Making European performance and impact assessment frameworks for research infrastructures glocal

Sustainability of research infrastructures (RIs) is a big challenge for funders, stakeholders and operators, and the development and adoption of adequate management tools is a major concern, namely tools for monitoring and evaluating their performance and impact. BioData.pt is the Portuguese Infrastructure of Biological data and the Portuguese node of the European Strategy Forum on Research Infrastructures "Landmark" ELIXIR. The foundations of this national research infrastructure were laid under the “Building BioData.pt” project, for four years. During this period, performance and impact indicators were collected and analysed under the light of international guidelines for assessing the performance and impact of European research infrastructures produced by the European Strategy Forum on Research Infrastructures, the Organisation for Economic Co-operation and Development and the EU-funded RI-PATHS project. The exercise shared herein showed that these frameworks can be adopted by national RIs, with the necessary adaptations, namely to reflect the national landscape and specificity of activities, and can be powerful tools in supporting the management of RIs. “Not everything that counts can be counted, and not everything that can be counted, counts”. (Attributed to William Bruce Cameron)


Introduction
The long-term sustainability of Research Infrastructures [1] (RIs) is of great importance to the European Strategy Forum on Research Infrastructures (ESFRI) and the European Union more broadly, as shown by calls for RIs to demonstrate their economic and wider benefit to society. 1 For the Organisation for Economic Co-operation and Development (OECD), sustainability is also a major concern as RIs represent an increasingly large share of research investment by national governments. 2 As a result, recent years have seen the emergence of a number of frameworks (ESFRI, OECD and that developed by the EU-funded RI-PATHS project) to guide RIs in their journeys to demonstrate performance and impact, going beyond simply scientific impact, and considering public value more generally.

Motivation
BioData.pt is the Portuguese Infrastructure of Biological Data and the Portuguese node of ELIXIR. Anticipating external pressure from funders, government or citizens, BioData.pt, under the internal leadership of Ana Portugal Melo as Executive Director, initiated this study as an instrument to evaluate the impact of public investment in such a research infrastructure, and support decision making in further funding to this and other RIs. The goal of this study was to consider three frameworks for impact and performance evaluation (ESFRI, OECD and RI-PATHS) in relation to existing indicators maintained by the BioData.pt project, from now on referred to as "Building BioData.pt", funded via the Portuguese state budget and European structural funds. From this, we expected to gain a more systematic, structured, and deeper understanding of the performance and impact of this national-level RI, and to use this new knowledge to inform its further development towards long-term sustainability. Beyond the qualitative assessment itself, this exercise aimed to document a process that may help national nodes of distributed European RIs to understand the impact of their activities, to inform stakeholders, policy makers and funders, as well as to improve their operations and increase their visibility, and overall prestige. In addition, it aimed to assess the relevance and adequacy of the indicators used by BioData.pt. It is expected that our findings, including lessons learned, will be useful to other similar public-funded RIs, which are often working with limited resources and do not have the means to fund repeated impact evaluations by specialized consultancies.
Organization and historical path Although BioData.pt is the Portuguese Infrastructure of Biological Data and the Portuguese Node of the ESFRI Landmark ELIXIR (a pan-European research infrastructure for life science data, Ref. 3), it relies on fixed-term project funds to operate. A historical note of BioData.pt and the Portuguese Node of ELIXIR is depicted in Figure 1.
Since 2017, the development and early operation of BioData.pt have been funded through a €2.7M project grant to last for four years, to lay the foundation of this RI. The "Building BioData.pt" project involved 11 beneficiaries and 1 According to the European Commission, research infrastructures are facilities that provide resources and services for the research communities to conduct research and foster innovation in their fields. These include. major equipment or sets of instruments. https://ec.europa. eu/info/research-and-innovation/funding/funding-opportunities/funding-programmes-and-open-calls/horizon-europe/research-infrastructures_en REVISED Amendments from Version 1 This version has a different title, correcting a previous lapse: "Making European performance and impact assessment frameworks for research infrastructures glocal".
The main modification of its content is a larger and deeper discussion of the management implications of the study performed herein, including a more detailed description of the overall process of indicator collection and analysis and the managerial structure within which it was conducted.
In detail: The "Discussion section" was modified to also reflect on the operational phase of BioData.pt beyond the "Building BioData. pt" project. The adaptability of the three impact assessment frameworks conceived for European RIs at the national node level was further specified, namely respecting the relevance of the different indicators and their adaptation from global level to a glocal context. The managerial process was further detailed, namely the leadership, the size and profile of the team that perform the study at its different stages, the benefits of a participatory approach and also recommendations for future similar exercises were provided. In the 'Methods section,' the procedure and criteria for scoring the indicators was further detailed. An overall proofreading was performed.
Any further responses from the reviewers can be found at the end of the article 81 participants, many of which providing in-kind expertise on computing, bioinformatics and data management to a range of end-users, as well as building and operating the infrastructure itself. Of the project budget, 33% was used to hire human resources and 57% to purchase equipment, mostly computing resources to assemble the nationally distributed computing infrastructure. The remainder was used in the adaptation of buildings to host the computing infrastructure or to deliver training. "Building BioData.pt" was structured as shown in Table 1.

Methods
Based on the simplified RI-PATHS approach, which classifies indicator types in activity (concrete activities to be carried out within the scope of the project, visible to the public and under full control of the organization), outcome (short-term direct results of each activity, not under direct control of the organization) and impact (transformative effects of the activity on its target audience and beyond, in the mid to long term), 4 a "BioData.pt Monitoring and Evaluation Matrix" (B-MEM) (Figure 2, Ref. 5) and supporting guidelines were developed to facilitate the onset of evaluation processes, by the BioData.pt infrastructure. This simple and participatory approach allowed the project management team (the project manager and the executive director) to view and document, for the first time, the 10 working groups (or work packages), involving about 50 researchers (Table 1) of "Building BioData.pt" in terms of activities, outcomes and impact, and align these with the overarching objectives of the project. In a second stage, the existing "Building BioData.pt" indicators, which had been maintained during the four years of grant execution (2017-2021), were categorized as activity, outcome and impact indicators, and assigned to the most relevant RI-PATHS impact area.
Finally, each existing "Building BioData.pt" indicator, which could be quantitative or qualitative by nature, was documented and cross-checked against those compiled by the RI-PATHS project, and the OECD and ESFRI frameworks.  The following part of this study involved 3 additional experts (2 from BioData.pt and 1 from ELIXIR Hub) to reflect and perform a thorough analysis.
Taking advantage of the previous exercise, and to assess the commitment of BioData.pt with its purpose as a national RI and the Portuguese Node of ELIXIR, and its potential contribution to international initiatives such as EOSC and the UN Sustainable Development Goals (SDGs), the categorised "Building BioData.pt" indicators or proxies were assessed against several sets of strategic objectives relevant to research infrastructures and their work: those of BioData.pt (as an organisation), those of the current ELIXIR Scientific Programme, 6 those of the Portuguese National Roadmap of Research Infrastructures from Fundação para a Ciência e a Tecnologia (FCT), 7 those of EOSC 8 and the SDGs. 9 Arbitrary units (A.U.) were attributed to each of these dimensions, according to the level of BioData.pt commitments: objectives/ goals of BioData.pt (4.5 A.U.), ELIXIR (2 A.U.), National Roadmap of RIs (2 A.U.), EOSC (1 A.U.) and the SDGs (0.5 A.U.), and a threshold was defined at 6.5 A.U., a score that could only be achieved if an indicator matched BioData.pt goals and, at least, those of ELIXIR or the National Roadmap of RIs, to which BioData.pt is bound by a formal relationship. Then, each indicator maintained during "Building BioData.pt" and categorised was analysed with respect to its contribution to the objectives of each of the former indicated organisations/initiatives.
To enable the visualisation of this analysis, a stacked column chart was generated entirely using the Rstudio software version 2021.09.0.351 and ggplot2 version 3.3.5, 10 specifically using the ggplot() function, due to the complexity of the graphic. The fct_reorder() function of forcats version 0.5.1 was used to reorder indicators according to their relevance, in decreasing order.

Implementation and impact analysis
The first step was the description of the "Building BioData.pt" project working groups, in terms of activities, outcomes and impact, and their alignment with its two overarching objectives, using the BioData.pt Impact Assessment Matrix (B-MEM) and the respective guidelines. Working groups (WG) 1 to 5 were dedicated to domain specific activities which, overall, aimed to build the BioData.pt Research Communities, and their expected outcomes were data curation, integration and availability for domain-specific areas with impact in enhancing the quality of scientific research by promoting the FAIR principles. The global activity of WGs 6, 7, 9 and 10 was to set up the Support to Research Communities, having three outcome/impact lines: 1) a fully available computational infrastructure for data analysis/ better and more extensive computing resources; 2) promotion of capacity-building in bioinformatics and research data management in the national research community/improved research efficiency & effectiveness and human capital; and 3) overall management of RI and operation of the Portuguese Node of ELIXIR/strengthened RI long-term sustainability. Both "Research Communities" and "Support to Research Communities" were developed under the objective "Strengthening research, technological development and innovation". The second objective "Enhancing research knowledge transfer from academy to industry" was served by the activity Industry Engagement, developed by WG8, delivering promotion of knowledge transfer to industry to bring an added value to companies' data and an industrial ecosystem of companies aware/beneficiary of bioinformatics and research data management as outcome and impact, respectively.
Yet with the help of the B-MEM, the "Building BioData.pt" project and Infrastructure reports were mined for indicators that would relate to those compiled by the ESFRI, OECD and RI-PATHS frameworks. Table 2 shows one example of this implementation, performed for the activity "Research Communities", in relation to RI-PATHS. In detail, for this activity, we were able to find indicators of two types, activity and outcome, distinctly distributed by three of four impact areas, Human Resources, Economic and Innovation and Societal/Social. For each type of indicator identified, the indicator itself, the evidence for confirmation and method for description is listed. For instance, under the objective "Strengthening research, technological development and innovation", in the activity "Research Communities", the activity indicator Number of publications is identified by direct evidence (e.g., from Web of Science) and the method for collecting evidence is counting the number of publications. The full dataset of BioData.pt project indicators classified following the RI-PATHS approach can be accessed. 11 This exercise was repeated using the impact assessment frameworks of ESFRI and OECD. These approaches cover more explicitly the so-called scientific impact, of which some indicators, like Number of publications and Number of Citations, are also covered in the Human Resources area of the RI-PATHS approach.
A summary of RI-PATHS' indicator types produced by "Building BioData.pt" (Table 3) shows that the different activities of the project, "Research Communities", "Support to Research Communities" and "Industry and Entrepreneurship", encompassing its 10 working groups had impact in all four RI-PATHS areas, namely, Human Resources, Economic and Innovation, Social/Societal and Policy, the working groups that provide "Support to Research Communities" having the broader socio-economic impact.
The indicators of "Building BioData.pt" identified in the ESFRI list corroborated the previous finding that the activities with broader impact were those dedicated to the "Support of Research Communities" ( Table 4). The detailed information for this exercise is available for consultation. 11 In addition, "Building BioData.pt" generated indicators for all ESFRI's areas and covered all indicators except number 14 (Number of publicly available data sets used externally).
The analysis of "Building BioData.pt" indicators using the OECD framework (Table 5) also corroborated the broad impact of the project, by retrieving indicators for all impact areas. In this case, the scientific impact produced by the activities of "Research Communities" assumes particular relevance. Nonetheless, as for the previous frameworks, the activities related to "Support to Research Communities" have a wider impact. The full dataset of this analysis can be consulted. 11 The indicators generated by "Industry and Entrepreneurship" activities could find matches or proxies in the lists of the three impact assessment approaches, with greater emphasis, as expected, in the Economic and Innovation impact areas, enriching the project with a translational dimension.
Overall, this analysis showed that the activities carried out to accomplish the "Building BioData.pt" overarching objectives of "Strengthening research, technological development and innovation" and "Enhancing research knowledge transfer from academy to industry" covered the broad range of impact areas foreseen for European Research Infrastructures. This not only puts this RI of the National Roadmap aligned with sibling international counterparts, but also suggests that BioData.pt is well aligned with ELIXIR, bringing to the national scientific system the state-of-the-art topics of computing, bioinformatics and data management, addressed by this ESFRI Landmark.  √ n/a n/a S10 -Structuring effects of the RI on the scientific community √ √ n/a S11 -Papers co-authored with regional universities √ n/a n/a S12 -Use and production of open data √ n/a n/a S13 -Data openness

Discussion
The RI-PATHS project listed 102 possible indicators to be considered when planning or carrying out performance and impact evaluation of RIs from different scientific areas (Health & Food, Data, Computing and Digital Research, Energy, Environment, Physical Sciences & Engineering, and Social & Cultural Innovation), and types (distributed or singlesited). RI-PATHS also proposes a set of pathways tailored to assess the socio-economic impact of specific project activities, encompassing smaller subgroups of indicators. For instance, pathways 5 -Learning and training by using RI facilities and services, 9 -Provision of specifically curated/edited data, and 11 -Creating and shaping scientific networks and communities seem better suited to evaluate the impact of BioData.pt training activities (e.g., "Ready for BioData Management?"), the CorkOak DB portal, 13 and the "building and consolidation of our Communities and Platforms, in a multidisciplinary network to identify gaps in computing data management and bioinformatics for the life sciences and build solutions".
From the referred universe of 102 RI-PATHS indicators, 25 were identified among those monitored during "Building BioData.pt", which scatter in all impact areas, suggesting the broad socio-economic impact of BioData.pt. When selecting RI-PATHS pathways more appropriate to assess BioData.pt activities, the indicator retrieval ratio increases to~50% of the corresponding indicators subgroup. This corroborates that the performance and impact indicators used to evaluate a specific project or activity need to be tailored to several parameters, namely the nature of the activity under assessment, e.g. a training program, a community, a data portal, and also a particular context such as a national RI, or a specific project, e.g., with a wider scope that covered simultaneously Enabling Science, Problem-Solving, and Science and Society, as in the case of the establishment of a national RI or to a particular activity with a narrower scope, in addition to the scientific domain or the entrepreneurial nature of such RI or activity.
Of the 21 indicators compiled by the ESFRI framework, 20 were monitored by "Building BioData.pt". The missing indicator, Number of publicly available data sets used externally, relates to the optimisation of data use and is a relevant indicator for a RI like BioData.pt. Although BioData.pt has publicly available datasets (e.g., Pheno and CorkOakDB 13 ), it does not currently maintain indicators to demonstrate how these are being used externally. This difficulty also intricates with the fact that there is still a long way to go in the formal acknowledgement of RI services by its users, which is harder when the goods provided are intangible and sometimes provided by third parties. In the future, this topic should be addressed by putting in place a tracking system. The ESFRI framework, currently with a very broad scope for each indicator, is more prone to embed the specificities of national RIs.
Finally, out of the 58 indicators compiled by the OECD framework, 32 were monitored by "Building BioData.pt", 18 belonging to Scientific Impact and Training and Education, emphasizing the importance of these areas for BioData.pt. In this framework, there is a large collection of patent and industry related indicators that, for the time being, are not part of BioData.pt objectives. There is also a set of indicators proposed by OECD related to the European geography, that could be adapted to the national landscape (e.g. T24 -Regional firms funding RI).
It is noteworthy to remember that this exercise did not benefit from an ex-ante impact evaluation and therefore the indicators maintained were not previously selected to monitor future socio-economic impact but the accomplishment of the funders' requirements (Portugal 2020 program and Fundação para a Ciência e a Tecnologia). Nonetheless, the three frameworks considered in this study were comprehensive enough to accommodate all indicators maintained during "Building BioData.pt", in spite of the funding instruments, specific national context, nature of RI activities and role as a national node of a European RI. 14 In addition, absent indicators may also prove useful to bring our attention to untracked activities that are relevant to the RI mission of an established national RI. One example is the long-term tracking of human resources in the RI, namely trainees, and scientific and managerial staff, their stays in the RI and their careers after leaving.
The mechanism for such tracking is not trivial for a distributed organisation. Another example could be the already mentioned adaptation of indicators to monitor regional collaborations/networks and projects, or suppliers, and expenditure and income to the national scale. In the particular case of BioData.pt, which is now a legal entity, monitoring the number of its associates and the ability to capture users paid by the national funders, via time or service vouchers, would enable to assess the contribution of these activities to the RI sustainability.
The vast majority of the ESFRI, OECD and RI-PATHS indicators identified in "Building BioData.pt" surpassed the 6.5 A.U. in our scoring exercise, indicating that, this project was well aligned with current thinking around the performance and impact of RIs. This also reflects the natural integration of BioData.pt in the European and Portuguese research landscape. Furthermore, alignment with the EOSC goals was also very frequently observed underlining the commitment of BioData.pt with the best practices of FAIR data management and open science. It was interesting to observe the overlap between BioData.pt and the SDGs, suggesting that national RIs, as likely their European counterparts, can give a valuable contribution to global sustainable development, and are thus a public good. Below the threshold, three indicators contributing to the strategic objectives of the BioData.pt infrastructure were identified that did not promote the ELIXIR priorities or those of the National RoadMap of Research Infrastructures. These were: Number of employees, Income from commercial activities and number of entities paying for service, and Visitors on website and followers on social media/ Outreach via the RI's own web and social media. Although these indicators are not directly related to the mission of BioData.pt, they may assume a very relevant role for its long-term sustainability and thus, perhaps both the National Roadmap for Research Infrastructures and ELIXIR would like to consider this information when revising their priorities.
The process of collecting and categorising the indicators was well received by the project managers, who after a learning curve to apprehend the context and content of the new process, gained the understanding of the importance of evaluating the performance and impact of research infrastructure activities and the range of this potential impact. In detail, the project managers continuously collected and maintained the indicators generated by BioData.pt teams, which were reported annually to demonstrate the progress of the project. The participatory approach used to categorise the indicators, the B-MEM, allowed a wider participation of the infrastructure management staff in the conversion of a set of indicators in an impact analysis. The latter also enhanced the motivation and commitment of the managerial staff with the mission of BioData.pt. In a future opportunity, this process could be widen to the research/technical teams, enabling them to consider ex-ante the potential changes their activities will foster, and gather their contribution to the definition and collection of indicators. The wider vision gained by these teams is expected to create greater receptivity for the necessary indicator collection task, and also enhance their enthusiasm with their projects.

Final remarks
The monitoring of RIs performance and impact through indicators can be powerful in strategic management not only for continuous monitoring but also to inform future steps at the level of funders, optimization of processes and visibility by users and citizens. 15,16 Although highly qualified consultants are available to provide such service, national RIs are typically underfunded, and cannot support the cost. It is thus critical to establish simple and participatory methods to be implemented by the RI staff with oversight from the management team.
From this study, it is clear that the approaches developed by ESFRI, OECD and RI-PATHS are very useful and adaptable to the national context. It is also clear that special attention should be given to the national and thematic context of the activities under assessment. It is, thus, critical that each RI maintains the indicators that are more suitable to demonstrate the impact of its activities.
Overall, there is a strong alignment of "Building BioData.pt" with the mission of a research infrastructure as foreseen by ESFRI. The major flaw recognised under this study, respecting the difficulty to control external use and acknowledgement of RI assets, such us datasets, may need a collective solution to be addressed in the future.
This exercise was done backwards to provide lessons for the future, where ex ante the areas where impact aims to be achieved, the different types indicators for each activity to be collected, as well as the methods for capturing the indicators should be documented.
In this scope, the methodology used to assess the relevance of the indicators maintained during "Building BioData.pt" (Figure 3) can be useful to adapt the selection of indicators to a specific context, taking into account the mission and goals of the RI, and also those of national and international organizations and global initiatives composing the landscape where the RI develops its work and aims to have impact. To successfully assess performance and impact of RIs it is also very important to set up processes where the staff should be involved already from the planning stage, such as the BioData.pt monitoring and evaluation matrix (Figure 2). Such methodology permits the involvement of managerial and research/ technical staff and incorporates both perspectives. Moreover, RI staff gains a wider view of the organisation and feels more committed with its success and motivated to contribute in the collection of the indicators in the long-term. This paper examines how the international frameworks developed for the monitoring and evaluation of performance and impact of research infrastructures (RIs) can be adopted by national RIs to support its management. This is done by presenting the results of a four-year project of collecting and analyzing performance indicators carried out during the development of the Portuguese Infrastructure of Biological Data (BioData.pt) as the national node of an international RI (ELIXIR). The authors focus on three performance measurement frameworks developed for RIs (ESFRI, OECD, and RI-PATHS) and empirically assess how the combination of suggested performance indicators can be applied at the operational level to assess the impact of the organization. Within this perspective, the authors aim to contribute to developing useful tools for RI management.

Data availability
The article is clear and well written, however, it needs improvement. Our main point is that the authors could better discuss the management implications of their findings. Below we present our main points and suggestions: Firstly, the authors focus from the beginning on the process of development of the national node -the 'Building BioData.pt' project -paying scarce attention in the Discussion section to the operational phase, for example, which indicators can also be relevant for the permanent organization/node itself rather than for the project phases. We would encourage the authors to clarify this from the beginning since the abstract seems to propose general implications for RI management. We would like to see some reflections at a general level (maybe also in the conclusions).

○
One major point that needs further specification is, first of all, the adaptability of the three frameworks at the national node level. The authors present a list of 77 retrieved indicators out of 181 from the three frameworks. It is still not clear why some indicators were more relevant than others, and why many of the proposed indicators were excluded. The authors need to better clarify if this was related to the need to adjust the impact indicators to the context of the project, to the construction stage of the RI, to the national goals/strategy, or to data availability. Thus, the authors need to discuss in more detail how the case studied and the findings might affect the generalisability of the results. In other words, the issue of adaptation from 'global' to 'local' needs to be better addressed (in the discussion/conclusion).

○
Another point that requires further specification is not only the relation between the indicators and the national context/strategy but also the managerial implications of collecting and using these indicators. Apart from the issue of collecting data from third parties, we think that the process of staff engagement in the measurement process should be discussed further. For example, the authors could consider the opportunity to provide more insights into how the participatory approach was conducted and received by the staff. The engagement of team members in performance measurement could be an important aspect to be emphasised as a management implication.
In the 'Methods section,' the authors report the results of scoring of indicators to assess their alignment with national and international goals. No information is provided about the method and reliability of the scoring process. Moreover, it is not clarified what is meant here with the concept of 'alignment'.

○
Since the article focuses on RIs, we would encourage the authors to include it in the title.
○ Table 1. The activity "Industry Engagement" is mentioned as "Industry and Entrepreneurship" in the 'Implementation and impact analysis' section.  The authors have read thoroughly read your appreciation of this article and proceeded to the necessary modifications. We are very grateful for your analysis and think that it helped us to improve the quality of the article.
In detail: Firstly, the authors focus from the beginning on the process of development of the national node -the 'Building BioData.pt' project -paying scarce attention in the Discussion section to the operational phase, for example, which indicators can also be relevant for the permanent organization/node itself rather than for the project phases. We would encourage the authors to clarify this from the beginning since the abstract seems to propose general implications for RI management. We would like to see some reflections at a general level (maybe also in the conclusions).
○ A paragraph was added to the discussion to suggest a set of indicators that can be relevant for the operational stage of BioData.pt, as a mature infrastructure. One major point that needs further specification is, first of all, the adaptability of the three frameworks at the national node level. The authors present a list of 77 retrieved indicators out of 181 from the three frameworks. It is still not clear why some indicators were more relevant than others, and why many of the proposed indicators were excluded. The authors need to better clarify if this was related to the need to adjust the impact indicators to the context of the project, to the construction stage of the RI, to the national goals/strategy, or to data availability. Thus, the authors need to discuss in more detail how the case studied and the findings might affect the generalisability of the results. In other words, the issue of adaptation from 'global' to 'local' needs to be better addressed (in the discussion/conclusion).

○
The discussion was reformulated to address and clarify the adaptability of the frameworks to the national node level, and the criteria to discriminate and attribute relevance to the indicators. Overall, the discussion section is now more detailed and emphasises the issue of adaptation from 'global' to 'local' needs. Another point that requires further specification is not only the relation between the indicators and the national context/strategy but also the managerial implications of collecting and using these indicators. Apart from the issue of collecting data from third parties, we think that the process of staff engagement in the measurement process should be discussed further. For example, the authors could consider the opportunity to provide more insights into how the participatory approach was conducted and received by the staff. The engagement of team members in performance measurement could be an important aspect to be emphasised as a management implication.

○
The managerial implications of using the selected indicators are described, and the process of staff engagement is documented including recommendations for future assessments. In the 'Methods section,' the authors report the results of scoring of indicators to assess their alignment with national and international goals. No information is provided about the method and reliability of the scoring process. Moreover, it is not clarified what is meant here with the concept of 'alignment'.
The methods section was modified to clarify the concept of "alignment" and further information about the method and scoring process was included.
Since the article focuses on RIs, we would encourage the authors to include it in the title.

○
The title was modified accordingly.

○
The table was modified accordingly. Table 1. WG9 is missing.

○
The table was modified accordingly. Table 1. The activity "Industry Engagement" is mentioned as "Industry and Entrepreneurship" in the 'Implementation and impact analysis' section. indicators.
The main strengths of the article are related to the use of wide-spectrum frameworks created for the analysis of pan-European and global research infrastructures to a national (Portuguese) research Infrastructure, with ad hoc adaptations to reflect the national landscape and specificity of activities. The article also underlines the possibility of conducting a backward exercise able to prospect recommendations for the future, determine areas where impact aims are to be achieved, the indicators for each activity to be collected, as well as the methods for capturing should be documented.
A few remarks: The title "Making European performance and impact assessment frameworks glocal" does not allow inferring the content of the typescript. Although the article is inserted in the ELIXIR gateway, I suggest that reference be made to the performance and impact of BioData.pt according to the addressed frameworks (ESFRI, OECD, RI-PATHS project).

○
One last note: the quote "Not everything that counts can be counted, and not everything that can be counted, counts", is frequently incorrectly attributed to Albert Einstein. Probably comes from a paper published in 1963 by the sociologist, William Bruce Cameron (Cullis 2017 1 ). ○