Research Software vs. Research Data I: Towards a Research Data definition in the Open Science context

1. Introduction

Each particle of the Universe, known or unknown by what is widely accepted as Science, is information. Different datasets can be associated to each particle to convey information, as, for example: where has this particle been discovered? By whom? At what time? Is this particle a constituent element of a rock, or a plant, or…? Indeed, as living entities of the Earth planet, …we are all part of this Universe and every atom in our bodies came from a star that exploded…, therefore …we are all stardust….¹

So long ago that we have never been able to give a precise date, information started to be fixed in cave paintings, figurines, and bone cravings, which have been found in caves like Altamira² or Lascaux³. That is, some human beings intentionally fixed information on a support. Much more recently, languages have been developed to deal with information, fixing and exchanging it in clay bricks, papyrus, monument walls, and paper books. Even more recently, information has been fixed in films, photographs, and has finally adopted digital formats.

Scientists study all kinds of subjects and objects: persons, animals, trees and plants and other living beings, philosophies and philosophers, artists and artworks, mathematical theories, music, languages, societies, cities, Earth and many other planets and exoplanets, clouds, weather and climate, stars and galaxies, as well as other animate or inanimate objects, molecules, particles, nanoparticles and viruses, nowadays including digital objects such as computer programs. Some of these items, like images, texts, and music etc. may have associated intellectual property rights; but others, like statistics or geographical data, may not. Yet, they may be affected by other legal contexts, such as, for example, the one given by the EU INSPIRE Directive 1 for spatial data, concerning any data with a direct or indirect reference to a specific location or geographical area.

Now, in our digital era, most of the above subjects under consideration are handled by humans using computers, through numerical data. Scientists present new theories and results built and produced with numerical simulations and through the analysis of numerical datasets. They are usually stored in databases, manipulated or produced in digital environments using existing software, either Free/Open Source Software (FLOSS)⁴ or commercial, or by means of software developed by research teams to address specific problems 2,3.

In this specific scientific context, the aims and developments of Open Science practices are particularly relevant. Indeed, as remarked by 4: "We must all accept that science is data and that data are science…". Therefore, in this article we take into consideration the following definition of Open Science, in which the open access to Research Data (RD) and to Research Software (RS) is part of the core pillars 5:

A more transversal and global vision can be found in the ongoing work for the UNESCO Recommendation on Open Science⁵ 6. See also 7 for another relevant example of ongoing work on the Open Science concept.

Among the most important kinds of research outputs of any scientific work, we focus on the trio formed by articles, software and data. Actually, among all the possible duos, the couple RS and RD present more similarities. For instance, unlike the dissemination of published articles, usually at the hands of scientific editors, the dissemination of software and data that have been produced in the research process is mostly at the hands of their producers, the research team. More generally, as sketched in 8, the release of RD and RS raises the same questions, at the same time, in the production context. As a direct consequence, it seems suitable to propose a similar dissemination procedure 9.

As mentioned in the last reference, both RS and RD can be disseminated using licenses to set their sharing conditions. RS licenses and licensing information can be found at the Free Software Foundation (FSF)⁶, the Open Source Initiative (OSI)⁷, and the Software Package Data Exchange (SPDX)⁸. The SPDX licenses list also includes licenses that can be used for databases, like the Creative Commons licenses⁹ or the Open Data Commons Licenses¹⁰. On the other hand, let us remark that Data Management Plans are nowadays required by several research funders (see for example 10,11) and that Software Management Plans are also available 12. Finally, concerning evaluation, as observed in 3, a similar evaluation protocol can be proposed for both RS and RD. These two subjects, RD dissemination and evaluation, are more closely analyzed in the article 13 that follows the present work.

Inspired by this collection of analogies, we will argue in the next sections that a definition for RD can be proposed following the main features of the RS definition given in our recent work 3,14. However, it remains a challenging issue. In fact, although one of the most widely accepted RD definitions is the one proposed by the OECD (2007) 15, other works have shown the difficulties to fix such a definition 16,17.

Indeed, this concept has important and not well settled consequences, for example, in the context of data sharing, as highlighted by C. Borgman in 18:

It is the intention of our present work to bring some answers to these questions.

The plan of this article is as follows. The next section introduces the concept of RS after a summary presentation of the key points involved in the notion of software as a legal object. Section 3 is devoted to discuss the different issues involved in the challenge towards a precise data definition. Section 4 describes partially the landscape of existing work addressing the RD definition, enumerating some difficulties to settle such a concept. Then we propose our RD definition based in three characteristics: the data should be produced (collected, processed, analyzed, shared & disseminated) to answer a scientific question, by a scientific team, and has yield a result published or disseminated in some article or scientific contribution of any kind. Comparisons with other RD definitions are examined. The last and final section concludes with the proposition of some specific answers to Borgman’s conundrum challenges 18. Let us remark that these conundrum challenges involve as well RD dissemination issues that are studied in detail in the article that follows this work 13, which also includes the analysis of RD evaluation issues.

The reader of the current work should be aware that its authors are not legal experts. Thus, in order to complete this article, we have analyzed legal documents and articles written by law experts 1,16,17,19–31, but from the scientist’s point of view. Yet, a deeper understanding of legal issues may require the intervention of legal specialists.

Following the standard scientific behavior (the authors of this work are mathematicians), we have detected a problem – the need to provide a more suitable RD definition – and have observed the involved landscape and studied the related literature; we have focused on and structured different components of the problem; finally, we have proposed what we think can be a solution for the challenge under consideration. As in any other research work, we believe that our proposal should be examined by the scientific community in order to evaluate its correctness, and to help improving it, if needed, advancing towards a better solution.

2. Research Software

In this section we bring together some of the existing definitions of software as a legal object (see references below). We also recall our definition of RS coming from 3,14.

3. The challenges of a data definition

As stated in 36:

Unlike software, data is, as a legal object, much more difficult to grasp. In fact, according to 30, data is not a legal concept, as it does not fall into a specific legal regime. For example, data can be either mere information or une œuvre, a work with associated intellectual property, when it involves creative choices in its production that reflect the author’s personality 29. The Knowledge Exchange report 17 provide guidelines that can be used to assess the legal status of research data, and mentions:

In relation with this legal context of data sharing and reuse, a very complete framework is introduced in 19:

Another list of legal issues related to data is provided by 30, similar but not equal to the one in the previous quote.

Yet, it is also necessary to consider other legal contexts concerning, for example, les données couvertes par le secret médical ou le secret industriel et commercial ¹⁹. Let us remark that the section Applicable Laws and Regulations of 11 provides a broad overview of regulatory aspects that need to be taken into consideration when developing disciplinary RD management protocols.

The problem is that data can refer to many different subjects or objects and they may be protected under different laws. It is not our intention to consider here these legal aspects. We need to simplify the context to help us to set a manageable concept of research data in the scientific framework. For this purpose we present here two relevant data definitions found in the data scientific literature.

The OECD data definition in its Glossary of Statistical Terms²⁰ states that:

Also, as a relevant precedent, let us quote here the data definition of the Committee for a Study on Promoting Access to Scientific and Technical Data for the Public Interest, as mentioned in 37:

To better grasp the connection between the concepts of data and information we have consulted several sources of different natures, as for example: the definition of dato²¹ and the definition of información²² in the Diccionario de la lengua española of the Real Academia Española, the definition of donnée²³ and the definition of information²⁴ in the Dictionnaire Larousse de français, and the data definition²⁵ in Wikipedia. Note that we can find information among the data synonyms in the Larousse dictionary, but data is not among the information synonyms. On the other hand, Wikipedia mentions that both terms can be used interchangeably, but that they have different meanings. These sources bring to us an eclectic panorama on the ingredients that could form a data definition and their relation with the concept of information. Some extracts of the texts in the links mentioned in the previous footnotes have been included in Box 1.

Yet, the above considerations only provide a light panorama of the complexity that is hiding behind the concepts of data and information. To illustrate this complexity in full terms we can refer to the French Code de l’environnement 26. In its Article L-124-2²⁶ we can appreciate the subtleties of the definition of environmental data in the following description:

To be compared with the much more easier to understand concept of geographical data as introduced by the Article L127-1²⁸ of the same Code de l’environnement 26:

Another example to show the complexity of the representation and manipulation of data and information that we would like to mention here corresponds to the linguistic research work developed at the Laboratoire d’informatique Gaspard-Monge, where one of the authors of the present work resides, see for example the doctoral thesis 38,39.

An additional factor that adds complexity to the concept of scientific data has to do with the potential use(s) and sharing of these data. As remarked by the OECD Glossary of Statistical Terms³⁰:

The importance of the context is also noted in 18:

This opens the door to a series of complex issues. For example, to the need for complementary, technical information associated to a given dataset in order to facilitate its reuse. See 40 (p.16) (and also 36) that highlights the difficulties raised by the concept of temperature related data, as explained by a CENS biologist:

Another instance to further illustrate the complexity of technical information associated to a data set in the STRENDA Guidelines that have been developed to assist authors to provide data describing their investigations of enzyme activities.³¹ Other examples from the collection of complex issues associated to data use(s) and sharing conditions are:

finds a strong formulation for scientific data in 41:

For more information on ‘re-use’ see, for example, 16,21,29,40.

Let us remark again that legal aspects arise quite naturally in the above list of items. Among others, some aspects are related to the fact that the datasets are usually organized in databases, where data is arranged in a systematic or methodical way and is individually accessible by electronic or other means 16,17,20,24,28. The intellectual property rights can apply to the content of a database, the disposition of its elements and to the tools that make it working (for example software). The sui generis database rights primarily protects the producer of the database and may prohibit, for instance, the extraction and/or reuse of all or a substantial part of its content 20.

Finally, let us quote here this paragraph from the OpenAIRE project report 16 (p.19) that highlights the difficulties to set a research data definition in the context of legal studies:

4. Data as a research output: towards a definition for Research Data

After a partial description of the current difficulties involved in the RD concept, we propose here a RD definition, directly derived from the RS definition presented in Section 2.2. To this aim we start by gathering some previous definitions that are particularly relevant for our proposal.

The first one is the White House document 31, and in particular the Intangible property section where we can find the following definition.

Let us remark that, according to31 this definition explicitly excludes:

The above RD definition has been extended in 46, emphasizing, among other aspects, the scientific purpose of the recorded factual material and the link with the scientific community.

A second basic inspiration for our proposal is the Directive for Open Data 21 that states:

The third pillar that we consider essential to support our proposal is the OECD report 15 (p.13) where we can find one of the most largely accepted and adopted definitions of RD:

A remarkable “positive” aspect of these three definitions is that they separate the data from the subject under study, and set what is, or is not, RD. This is relevant, as the legal context of the subjects under study sets up the legal (and ethical) context of the RD.

We don’t agree completely with (for example) the exclusion of the laboratory notebooks as RD elements, as they can be used as input data for other studies (how a laboratory works, which is the information that appears in some notebooks depending on the scientific matter). We think that these information and data can be of interest for other researchers.

Some “negative” aspects: the role of the data producers does not appear in the definitions although it is more or less hidden under the connection with the scientific community. But their role is very important as observed in 40 (p.6):

Indeed, as for each research output, the producer team is the guarantor of the data quality, in particular to ensure that the data are not outdated, erroneous, falsified, irrelevant, and unusable. Note that this is particularly relevant in the case of RD, as a consequence of the lack of a widely accepted publication procedure as the one existing for articles in scientific journals, where the responsibility of the quality of the publication is somehow shared by the authors, the journal editors, and the reviewers. This is also confirmed by 47 (p. 73):

Moreover, note that, as remarked in 19 the quality of the producer legal entity defines the cultural quality of the data in legal terms, and thus we have cultural data.

Let us observe that the central role of the scientific team as the producer of the research output is missing in the preceding definitions, but it is distinctly highlighted in our RS definition (see Section 2.2), which implies that the quality of the producer research team reflects over the quality of the software.

On the other hand, in these three definitions, the RD scientific purpose is focused in its role to validate research findings, although RD can be reused for many other finalities in the scientific context as, for instance, to generate new knowledge, i.e. as primary sources for new scientific findings. These are two of the four rationales for data sharing examined in 18.

Bearing all this in mind, we propose the following RD definition.

As previously declared, we have followed closely the RS definition in Section 2.2, translating, as a consequence, the digital nature of RS to RD. This does not mean that we do not consider physical samples as possible RD, but rather we assume that the information extracted from the physical samples has been already treated as digital information to be manipulated in a computer system, which simplifies the manipulation of physical data and its inclusion in the proposed RD definition.

Thus, RD has three main characteristics:

Let us observe that the first one agrees with 36 (p. 508): … data from scientific sensors are a means and not an end for their own research.

Note that, according to our definition, documentation, licenses, Data Management Plans and other documents can also be part of the set of files that constitutes the RD. Moreover, as explained in Section 2.2, a RS can also include data in the list of included materials that could also be qualified as RD. There are here a broad spectrum of possibilities, according to the size, the importance given by the research team and the chosen strategy in the dissemination stage. If the RD is considered of little size and less importance than the RS, it can be just included and disseminated as part of the software, and also the other way around, when the RS is considered less important than the RD, as for example when the software development effort is much less important than the time and effort invested in the data collection and analysis. It can also happen that both outputs are considered as of equal value, and can be disseminated separately. In this case it is important that both outputs are linked in order to allow other researchers to find easily the other output.

In a similar manner as for RS, RD can include other data components, and the RD producer team should explain how these components have been selected, mixed and analyzed, and highlight the reuse of other RD components (with citations, see for example 33,35,37,48).

Moreover, software and data can have several versions and releases, and they can be manipulated alike and with similar tools (forges, etc…) 34,49,50. One of the differences that we have detected between RS and RD is that while some research teams can decide to give access to early stages of the software development, what we observe in the consulted work is that RD is expected in its final form, ready for reuse, as mentioned in 18:

This difference is a consequence of the distinct nature of the building process of both objects. In the FLOSS community, we find the release early, release often principle associated to the development of the Linux kernel 51 and to Agile developments.³⁵ This principle may not have the same sense in the building of a dataset for which a research team collects, processes and analyzes data with a very particular research purpose, maybe difficult to share with a large or external community in the early stages of the RD production.

Note also that, in this work, we do not consider production issues like best software development practices or data curation. In here, the research outputs have reach a status in which the research team is happy enough for its dissemination. Neither we do enter in the different roles (see 18) that may appear in the RD team, taking care of actions involving: collection, cleaning, selection, documentation, analysis, curation, preservation, or the role of Data Officer proposed in.¹¹

5. Conclusion

RS and RD present many similarities. Analysing the different aspects of what means to define software and to define RS has driven us to the proposition of a RD definition in an independent way of a data definition. As a side effect, the fact that we can adopt easily the RS definition formulation for RD confirms and validates the RS definition.

In the introduction we have mentioned Borgman’s conundrum challenges related to RD 18:

We think that the proposed RD definition provides some answers to these queries, as well as to two extra ones that we consider equally relevant, namely how and where to share RD:

Which data might be shared? Following the arguments supporting our RD definition, we think that it is a decision of the research team: similarly to the stage in which the team decides to present some research work in the form of a document for its dissemination as a preprint, or a journal article, a conference paper, a book… the team decides which data might be shared, in which form and when (following maybe funder or institutional Open Science requirements).

By Whom? The research team that has collected, processed, analyzed the RD, and decides to share & disseminate it, that is the RD producer team. Data ownership issues have been discussed for example in 16,17,29,52-54.

How? By following some kind of dissemination procedure like the one proposed in 9 in order to identify correctly the RD set of files, to set a title and the list of persons in the producer team (that can be completed with their different roles), to determine the important versions and associated dates, to give a documentation, to verify the legal 17,30 (and ethical) context of the RD and give the license to settle the sharing conditions 28, etc. which can include the publication of a data paper. In order to increase the return on public investments in scientific research, RD dissemination could respect principles and follow guidelines as described in 15,55. Further analysis on RD dissemination issues can be found in 13.

Where? There are different places to disseminate a RD, including the web pages of the producer team, of the funded project, or in a existing data repository. The Registry of Research Data Repository³⁶ is a global registry of RD repositories that covers repositories from different academic disciplines. It is funded by the German Research Foundation (DFG)³⁷ and it can help to find the right repository. Note that the Science Europe report 56 provides criteria for the selection of trustworthy repositories to deposit RD.

With whom? Each act of scholar communication has its own target public, and initially, the RD dissemination strategy can target the same public as the one that could be interested by the associated research article. But it can happen that the RD is of interdisciplinary value, larger than the initial discipline targeted by the publication, and this public can be difficult to assess at first glance.

As observed by 18:

So, it can be difficult to assess the target community of interest for a particular RD, but this also happens for articles, and it seems to us that this has never been an obstacle for sharing a publication. Thus 18:

Under what conditions? The sharing conditions are to be found in the licence that goes with the RD, it can be for example a Creative Commons licence³⁸ or other licenses to settle the attribution, re-use, mining… conditions 28. For example in France, the law of 2016 for a Digital Republic Act sets in a Décret the list of licenses that can be used for RS or RD release 29,27.

Why and to what effects? There maybe different reasons to release some RD, from the contribution to build more solid and easy to validate science to simply answer to the recommendations or requirements of the funder of a project, of the institutions supporting the research team, or those of a scientific journal, including Open Science issues 5. The works 41,18 give a thorough analysis on this subject. As documented there and already mentioned in Section 3:

As remarked in 5 and in the work analyzed there, the evaluation step is an important enabler in order to improve the adoption of Open Science best practices and to increase RD sharing and open access. To disseminate high quality RD outputs asks for time, work and hands willing to verify the quality of the data, write a documentation, etc. Incentives are needed to motivate the teams. It also asks for the establishment of best citation practices and evolution in the protocols of research evaluation. In particular, the CDUR protocol 3 proposed for RS evaluation can be proposed for RD as presented in the article that follows the present work 13.

Data availability