Training to act FAIR: A pre-post study on teaching FAIR guiding principles to (future) researchers in higher education.

The scientific community has tried to implement the FAIR guiding principles to foster open science actions in data-driven research in higher education since 2016. However, what strategies work and do not in fostering open science actions still need to be determined. This article is the first step to closing this research gap by examining one strategy, the effectiveness of FAIR training in higher education. With a pre-post test design, the study evaluates the short-term effectiveness of FAIR training on students’ scientific suggestions and justifications in line with FAIR’s guiding principles. The study also assesses the influence of university legal frameworks on students’ inclination towards FAIR training. Before FAIR training, 81.1% of students suggested that scientific actions were not in line with the FAIR guiding principles. However, there is a 3.75-fold increase in suggestions that adhere to these principles after the training. Interestingly, the training does not significantly impact how students justify FAIR actions. The study observes a positive correlation between the presence of university legal frameworks on FAIR guiding principles and students’ inclination towards FAIR training. The study underscores the training potential in driving the transition towards open science actions in higher education and shows how much university legal frameworks can push toward such training. Students rate FAIR training as very useful and satisfactory. Important learning factors in effective FAIR training seem to be creating a safe space, letting students contribute, and encouraging students to engage in the training. However, the study also reveals the need for further training improvement, particularly in enhancing students’ ability to justify FAIR actions.


Trial registration
is not applicable as this study involves no clinical trial.The study involves an educational intervention to improve students' actions and justi cations of the FAIR guiding principles in scienti c research.Kiel University (Germany) collected the open data in this study between June 2019 and November 2022.The institutional research committee (Central Ethics Committee of the University of Kiel) approved all procedures performed in the data collection with the approval number ZEK-10/20.

Background
Since 2016, the FAIR guiding principles have advanced the implementation of open science actions in data-driven research.These principles derive from Wilkinson et al.'s publication entitled The FAIR Guiding Principles for Scienti c Data Management and Stewardship [1], offering guidance for easy access to and exchange of digital research.
Organizations and projects worldwide -such as UNESCO's Recommendation on Open Science [2] -have referenced these principles and called to promote sustainable open science and reliable research.Research Integrity Advocates [3] and the Reproducibility Networks [4] with guidelines such as the Hong Kong Principles [3], the European Code of Conduct for Research Integrity [5], the Marseille Declaration [6] or scienti c standards for journals [7] are also parts of the movement around the FAIR guiding principles.They, too, advocate and address the need to promote a responsible research culture and support sustainable data reuse to foster reliable research.
Findability, Accessibility, Interoperability, and Reusability (FAIR) are the four key pillars of Wilkinson et al. principles [1].Originally written for data publishers and stewards, they build upon Data Citation Principles [8] and contain 15 statements [1].The target group expanded from data management in the last seven years, and many others in the research system adopted the FAIR guiding principles.So, the principles started also to guide open science actions for (future) researchers.
The implementation of these principles seemed -already in 2018 -to be the future of rst-rate academic research [9].A paradigm shift towards enabling and using FAIR guiding principles promised -then and today -to facilitate and foster a more reliable knowledge exchange.The FAIR guiding principles support minimal standards for highquality data sharing, bringing bene ts to research (such as standardization, accountability, quality of scienti c ndings, reproducibility, and replicability) and society (such as citizen participation in science, sharpened science communication, and political decision-making).
As more data-intensive research develops, implementing FAIR in A) research infrastructures and in B) researchperforming organizations seems to be a prevailing and minimum goal for us all [9,10].
Given the importance of researchers as open science drivers, higher education and stakeholders alike have regarded teaching FAIR guiding principles as a preeminent mission [2,5,10].As, for example, the 2021 Max Planck study explicitly laid out [10], these "FAIR" skills should become part of higher education.
In 2019, the Horizon2020 Path2Integrity initiative developed free FAIR training material to take the necessary steps to implement FAIR as a standard in higher education and create a culture of FAIR sharing [11].Similar actions were taken by the FORRT [12], PRIM&R [13], the UK Data Service [14], the Economic and Social Research Council [15], the Embassy of Good Science [16], FairsFair [17], Parthenos [18], the Consortium of European Social Science Data Archives [19], and others.
Recent studies by Koterwas et al. [20] and Goddiksen and Gjerris [21] summarize various teaching objectives and methods in such training, pointing to dialogical approaches [20] and advocating for longer training to enable students' mindset to grow [21].Shanahan et al. [22] speci cally document the latest progress on how to teach FAIR, and Sefcik et al. [23]  proper data management and to explain and justify arguments for FAIR data management [25].

Methods
The Path2Integrity open data collection contains data from the Path2Integrity learning card program [11,27], including the data from the Path2Integrity FAIR training.Path2Integrity is an open educational program, free of charge and usable for all disciplines in higher education.The learning principles from the handbook describe the training standards [27].The program aims to teach students "how responsible research needs to be conducted in order to be reliable and thus useful for society" [27].
The Path2Integrity open data was collected through: The online P2I questionnaire [28,29], which focuses on scienti c action and justi cation referring to research integrity topics in the European Code of Conduct (such as FAIR guiding principles, research procedures, collaborative work, and research environment), and The online Path2Integrity feedback sheet [30] focuses on the learners' immediate reaction to training.
Path2Integrity collected the data between August 2019 and January 2022.(See references 31 and 32 for the data).
From the Path2Integrity open data collection (see Fig. 1), we selected the sample of students who voluntarily attended the international "FAIR training" (intervention group, n = 96 in pre-test) and a sample who lled out the questionnaire (control group, n = 418 in pre-test).
According to the Path2Integrity open data metadata, "the control group was collected in two rounds, mainly between March 2021 and January 2022.The students for the control groups were mostly European students whose educators embedded the [questionnaire] into their courses.Therefore, they were also contacted through their trainers.Due to differences in intensity and content of the courses and to increases in the quantity of the nonrandomized control group, both courses in research integrity, responsible conduct of research, good research practices, scienti c working, research ethics or related topics, as well as non-related courses, were included in the control groups.In total, we reached out to 864 trainers ... From these, 60 trainers allowed us to conduct [our questionnaire] within a total of 89 of their groups."[33].
The study's intervention group participated in a one-day, role-playing online training that included a 90-minute FAIR training with the standardized learning card M8 [25].Path2Integrity asked all students to ll out the questionnaires voluntarily.As Table 1 shows, this data was collected online at different times.

Table 1 Data collection before and after the training data collection informed consent
With the P2I questionnaire [29] at the beginning of each training (pre-test).
Attached to the survey and a condition to proceed to give data.
Attached to the survey and a condition to proceed to give data.
With the questionnaire on research integrity [29] directly after each training (post-test).
Attached to the survey and a condition to proceed to give data.
Path2Integrity collected the data and informed consent sheets anonymized via online surveys and stored them safely at Kiel University.To calculate the measures of this study, we used the published open data collection [31,32] with the support of the evaluation work package team.Completing the questionnaires was voluntary, and not all students took part in pre-or post-evaluation.Because Path2Integrity did not give any incentives to attend their questionnaires, we estimate that some students were tired and dropped out before collecting the post-test.(See below our comment on attrition rates.) Table 1 outlines that students in the intervention group were given version M of the P2I questionnaire [29] once at the beginning and immediately at the end of the training.The control group answered the same questionnaire [29] at the beginning and end of their no-FAIR training.Also, all students in FAIR training were asked to ll out the feedback sheet [30].
The P2I evaluation form M contains two FAIR-related questions (hereafter referred to as SPM8 and SCSM8), each with four possible answers.
In SPM8/A, students answered the multiple-choice question: "In his research project, Ali has collected a large amount of research data that he would like to make available open access in accordance with the FAIR guiding principles.To follow good research practices, Ali ensures that his data …" (Please choose only one of the following:) • A1: are described with rich metadata to be machine-readable.
• A2: are stored on FAIR foundation servers.
• A3: can be found in every database possible.
• A4: do not contain any information about sexual orientation.
In SCSM8/B, students answered the question: "Ali's decision (above) is in line with good research practices because …" (Please choose only one of the following:) • B1: it ensures reliable research results.
• B2: it ensures the equal treatment of all research data.
• B3: it is the duty of Ali to follow this process.
• B4: the legal framework governing universities requires it.
The sample size from the data collection [31] is as follows: Table 3 summarizes the data sample and its characteristics strati ed for each answer.The feedback sheet [30] refers in cases of (no-)FAIR training to the following eleven questions: Motivational factor: My participation in the (no-)FAIR training was encouraged by the trainer.
Instructional factor: For me, the (no-)FAIR training was adequately guided.
Safe space factor: I could express my opinion freely in the (no-)FAIR group.
Participation factor: I was able to contribute something to the (no-)FAIR group.
Appropriateness factor: The duration of the (no-)FAIR training was appropriate to me.
Comprehensibility factor: I clearly understood the task of the (no-)FAIR training.
Commitment factor: For me, the structure of the (no-)FAIR training was good to follow.
Satisfaction factor: I am satis ed with the (no-)FAIR training as a whole.
Trust factor: I would recommend the (no-)FAIR training to my fellows.
Usefulness factor: I have learned something useful in the (no-)FAIR training.
Practical relevance factor: I could connect the (no-)FAIR training with my everyday life.
Next to these two FAIR-related questions, the Path2Integrity feedback sheet dataset [32] shows that 95 students (n feedback ) answered these Likert scale questions.We recoded the answers with 2 being the positive end of the Likert scale, 0 being the neutral middle, and − 2 the opposing end.
To close the research gap on the FAIR training effectiveness, we hypothesized: 1. that FAIR training had a positive impact on both the suggested action and the justi cation of the students and would thus yield a clear shift in the response behavior of the students in the post-compared to the pre-testing towards the correct answers in the P2I questionnaire (A1 and B1, respectively) in the intervention group; 2. that a particular training that focuses explicitly on FAIR training is necessary to produce this shift (if present), and we should thus not be able to reproduce the former effect (if it is present) in the control group; 3. the legal framework of the associated universities of the intervention group may impact how students justify their actions.In the pre-test, students in the intervention group may choose to answer B4 over B1 (which is the expected answer).
In each case, we evaluated whether the FAIR training impacted the students' response behavior via Pearson's -chisquare test, with the null being that response behavior is independent of pre-and post-testing.In the case of hypothesis 3, in which we explicitly targeted the answer category B4, the data were rst collapsed over the B4 column to obtain a 2x2 table.We regard a p-value of less than 0.01 as statistically signi cant.In case of rejection, we planned to evaluate the source of the shift by looking at Pearson's standardized residuals of the t.
We regard standardized residuals with absolute values above 2 (above 3) as an indication that the respective cell has an impact (strong impact) on rejecting the null.As a measure of association and to evaluate the effect size of the shift towards the respective answer, we present the odds ratio for choosing the respective answer (A1, B1, or B4, respectively) over the other categories.
In the second step, we 4. contrast the learning factors of the FAIR training with the feedback of the learners [32].
We assess via a volcano plot which of the following learning factors students ranked as highly positive factors. In

Limitations
As described above, the study's data is from non-randomized students.We also detected that data from the intervention group "only" has two group indicators, G2 and G3, referring to two different trainers.
We consider this study to have an acceptable risk of bias for a quasi-experimental design, which may lead to indirectness of evidence and very little inconsistency (heterogeneity).The quality of evidence in our quasiexperimental design is lower than in randomized controlled trials.Nevertheless, this study uses a standardized instrument and controls more circumstances than usual using standardized training [27].Herein lies why this study is a reliable rst source towards closing the research gap on FAIR training in higher education.The following paragraphs comment on our prior assessment of possible factors affecting the quality of the study's evidence.
Because we used a quasi-experimental and not a randomized design, we accept that Path2Integrity did not blind the students.Path2Integrity partners experienced that the students were aware that there were other (no) FAIR training sessions.There was no cross-over effect because they could voluntarily choose to participate.We rate the effect of non-blinding as marginal and the risk of bias in this domain as very low.
In contrast, we have a medium risk of bias due to incomplete outcome data and missing participation (see Fig. 1).
The attrition rate is high but in a normal range for educational online studies [34].We outlined the dropout rate with students no longer available to participate in the study and being absent for the post-test in Fig. 1.
Furthermore, we did not preregister this study.However, we wanted to be as thorough as possible.To lower the risk of bias due to selective outcomes, we report all data and highlight the leading and meaningful results in the text and easy-to-read tables and graphics.We report our intervention and control group results using odds ratios.
Nevertheless, we have a medium risk of confounding factors within all groups.However, we decided to proceed with evaluating the FAIR training because otherwise, we would stay empty-handed.We accept this medium confounding factor risk in favor of demonstrating factors that may in uence FAIR training and can be controlled in future studies.
Overall, this study "only" assesses the effectiveness and observed factors of a 90-minute-long FAIR training.On the one hand, such a short training might be a limitation; on the other hand, a signi cant shift after a 90-minute session could and is a promising result.

Results
Regarding hypothesis 1: Is FAIR training effective?
The odds for students to suggest a scienti c action that aligns with FAIR guiding principles increases 3,75-fold after attending the FAIR training, leading to 46.7% of the students suggesting FAIR actions after training compared to only 18.9% before the training (see Table 4).The 99%-con dence interval of the odds ratio is (1.50, 9.37).The pvalue for rejection independence of pre-and post-testing is p = 0.0019 ( ).We regard both the effect size and the rejection of the test as signi cant.
Table 4: Marginals for pre-and post-testing Table 5 shows Pearson's standardized residuals of the model t.Here, the rejection of independence of the axis is the result of a shift towards scienti c action in line with FAIR guiding principles (A1) in the post-test from all distractor categories.A clear impact of the FAIR training is visible, diverting answers away from "are stored on FAIR foundation servers" (A2).

Table 5: Standardized residuals of the scienti c action
We also expected that students would modify how they justify their FAIR actions.The odds for students to justify their scienti c action to ensure reliable research increases by a neglectable estimated 1.09-fold after the training.The 99%-con dence interval of the odds ratio is (0.47, 2.50), and the p-value for rejecting independence of the response pattern to pre-and post-testing lies at p = 0.0553.The effect should thus not be seen as statistically signi cant.Notably, the p-value for rejecting independence is peculiarly low for this small effect size.Table 6 shows Pearson's standardized residuals of the model t.As column B1 shows, the deviation of the data to the null model is minimal in the B1 cells.In contrast, though, the large absolute values in the cells of column B4 (> 2.5) deviate from the data and the null model.These values document a discrepancy of well above two standard deviations.This discrepancy clari es why we see such a low p-value for the overall statistical test.Thus, the FAIR training not only had no impact on how students justi ed their FAIR action but in addition, the most substantial shift in the response behavior is not towards B1 ("it ensures reliable research results," the expected answer) but towards B2 ("it ensures the equal treatment of research data").The shift results in an odds ratio of 1.87 for answering B2 after course completion.The 99%-con dence interval for effect is (0.75, 4.67).This increase is notable despite the effect, in and itself, being measured as statistically insigni cant.See the discussion on Zeitgeist for a possible explanation of this nding below.
Regarding hypothesis 2: Is there an effect in the control group?
We could not reproduce the signi cant change towards FAIR actions from the intervention group in the control group, where students did not receive FAIR training.The control group received very similar training with a different (no FAIR) focus, and we can show there is an estimated odds ratio of 1.31 (99%-con dence interval is (0.72, 2.39), p-value 0.6433) for choosing an action that aligns with FAIR guiding principles.This effect leans far from statistically signi cant.
The estimated odds for choosing a justi cation for their action that aligns with FAIR guiding principles even decrease 0,99-fold (99%-con dence interval is (0.58, 1.70), p-value 0.3471).The effect here is also far from signi cant.
Regarding hypothesis 3: How is the relation between universities' legal frameworks and FAIR training?
We hypothesized that students from universities with legal frameworks on FAIR guiding principles choose to attend voluntary FAIR training.In this case, students in the voluntary FAIR training may justify their scienti c actions more often with the legal framework.Indeed, the odds ratio to justify their scienti c actions with a legal framework is 1.56 for students in voluntary FAIR training concerning students in the control group.However, the effect is not statistically signi cant (a 99%-con dence interval for effect is (0.30, 1.17), p-value 0.0472).
Regarding 4: Which learning factors can we observe in FAIR training?
Furthermore, the FAIR training received purely positive results in the immediate feedback.Speci cally, the "safe space factor" question was answered with the highest mean of 1.47 in the FAIR training, with 2 being the positive end of the Likert scale, 0 being the neutral middle, and − 2 the opposing end.The appropriate factor -"I found the length of the learning units appropriate."wasrated lowest, but still positive with 0.75 in the FAIR group.
The volcano plot (mean response on the x-axis and p-value of a symmetric one-sample t-test with H_0: the expected answer is neutral on the y-axis) displayed in Fig. 2 shows that all 11 learning factors from the feedback deviate positively from the neutral middle.To consider the multiple testing problem, we only regarded an effect with a -log10(p-value) of above -log (0,01/11) = 3,04 as statistically signi cant.Our ndings about FAIR training in a nutshell: • (Future) researchers' pro ciency needs improvement in suggesting a scienti c action aligned with FAIR.
• The 90-minute FAIR training shows high learning success for learners from different disciplines and quali cation cycles in higher education.
• Institutional incentives such as university guidelines on FAIR guiding principles may push students to register (even) in extracurricular voluntary FAIR training.
• Learners from higher education say they learn something useful in FAIR training.
First, a noticeable and unexpected result is the FAIR scienti c action score, which was 18.9%, rising to 46,7% in the post-test.Explanations for this success in FAIR training give new possibilities in nurturing open science actions while using little timely effort with a "big" impact.
However -and second -comparing the scienti c actions and justi cations of FAIR guiding principles, most students could only suggest a suitable scienti c action and no justi cation after participating in the FAIR training.
Justifying research actions requires deeper insights, skills, and a tting mindset.We link the stagnation of how students justi ed their actions with Goddiksen's and Gjerri's request that changes of mindsets take (training) time [21].
Third, we connect the unexpected increase in answer B2, "it ensures the equal treatment of all research data."with the use of Path2Integrity learning methods "role play" [11,27], capturing the Zeitgeist [20,24,35], we are currently living in.Role-play is a method to encourage cooperative learning, thus fostering perspectives from different people (D-diversity), fair treatment, access, opportunity, and advancement for all learners (E-equity), and inclusive decision-making processes (I-inclusion).At the same time, achieving equity and equal chances are objectives that are currently implemented not only in higher education reforms but also in society at large [35].In addition, roleplay also stands for the current shift from a teacher-centered approach to a student-centered approach to teaching [20,24], giving students more free space to develop their ideas and participate in group discussions [11,20].Due to the training methods [11,20,24] and current trends [35], students may have transferred this process from "how" they learned to "what" they learned.
Another training result from the Path2Integrity training supports our Zeitgeist explanation, the so-called "research procedure training" result from Path2Integrity [33], which also used role play as the main learning method.With training, 14% more students in the research procedure training also ensure equal treatment in research by their scienti c actions.Without training, only 4% do [30].Maybe implicit learning, which the training did not intend, induced the shifts towards "it ensures the equal treatment of all research data".This nding on how the answer "it ensures the equal treatment of all research data" increased in two 90-minute sessions re ecting the communal and equal learning process is critical and, in the future, needs to be assessed.
Fourth, looking at FAIR training gateways, universities with legal frameworks on FAIR guiding principles seem to push students to attend voluntary FAIR training.This nding points to the strategy that the FAIR guiding principles should either be part of the university regulations and result in sanctions for non-compliance or that FAIR training should become mandatory for (future) researchers a liated with the university since change is less likely to happen if using FAIR is voluntary.

Fifth, studentsFigure 1 Flowchart
Abbreviations-Findable, Accessible, Interoperable, Reusable eval13,14,15,17, approaches to teaching academic integrity in general.Furthermore, challenges, perceptions, and tools of FAIR guiding principles and open science are currently elucidated and described in many ways[12,13,14,15,17, 18, 19].Next to the general community effort to implement the FAIR guiding principles and the current broad educational offers of FAIR training, we (still) need to know what works and what does not.Pownall et al. [24] outlined this year (2023) that we need rigorous, thorough, and transparent evidence about teaching open and reproducible scholarship.This research gap on the effectiveness of FAIR training has been present since the principles' inception in 2016.That is why Path2Integrity decided in 2019 not simply to design and offer FAIR training but also to evaluate the success and speci cs of FAIR training.

Table 2
displays the professional, disciplinary, age, country, and gender distribution of the intervention and control group in the pre-test.

Table 2 :
Distributions of intervention and control group in the pre-test.

Table 3 :
Data sample characteristics for SPM8/A and SCSM8/B Question SPM8/A targets the students' scienti c action, whereas SCSM8/B aims at their justi cation in SPM8/A.Path2Integrity expected students to answer A1 and B1[28].The answers A2, A3, and A4 are mere distractors from A1.However, Zollitsch et al.[28]explain that in the case of SCSM8, the answers B2, B3, and B4 are different justi cation patterns.
sum, we probe the effectiveness of the FAIR training by analyzing the data from the Path2Integrity open data collection.By comparing the learning success of both the intervention control group, we assess if FAIR training is effective and what learning factors were rated highest.

Table 6 :
Standardized residuals of the justi cation Statistical signi cance is the case for all learning factors.The ve FAIR learning factors with the highest positive results above show that FAIR training works.FAIR training supports open science practices in higher education by implementing FAIR guiding principles.The ndings underline and support the current efforts to use FAIR training as a strategy. The