Training of Greek Public Administrators in Legal Knowledge Management by Using the Legislation Editing Open Software (LEOS)

The article describes the evolution of an innovative legal interoperability laboratory within the National School of Public Administration and Local Government in Ath-ens, Greece. While critically analysing its general scope, objectives and structure, the knowledge management perspective of an interoperable legal informatics solu-tion for handling big open legal data is brought into focus. In the context of the laboratory, the use of Legislation Editing Open Software (LEOS), a legal informatics tool that has been developed as a European Union ISA2 solution, is highlighted. Different aspects of and lessons learned from the training of public administrators are discussed while assessing this operations support system for potential use as a web-based authoring tool by the Hellenic public service. The evaluation of both the laboratory and the tool was conducted based on expert surveys, and possible upgrades are discussed to make the training of public administration more efficient.


Introduction and State-of-Play in the Field
The digital transformation of public administration made substantial steps forward, not least because of the development of legal informatics tools and standards and their implementation in real-world systems (Loutsaris & Charalabidis, 2020;Pagallo et al., 2018). In recent years, innovation in the public sector was supported by advancements in artificial intelligence and the emergence of disruptive technologies in general, which may have synergies with legal informatics technology (Sovrano et al., 2020;Wirtz et al., 2019). In particular, Koryzis et al. (2021) showed that legal technology is a promising candidate for the parliamentary workspace.
Linked with legacy information and communication technology (ICT) systems in governance institutions, legal informatics solutions can potentially advance their natural functions while having a sizable impact on society at large. A significant number of such systems have been developed and now exist in various degrees of maturity (prototypes, demonstrators, pilots, production) affecting all branches of government. Palmirani (2021) showcased a system to model the legal knowledge information from the Italian Official Gazette as well as the constitutional court decisions. The Austrian and the Hellenic Parliament were involved in designing and implementation of a system (ManyLaws) to retrieve, compare and visualise multilingual legal resources (Stavropoulou et al., 2020). Notably, legal informatics systems can positively influence the field of justice (see, e.g. Sharma et al., 2021) and Sharma et al. (2022), for instance, have implemented a legal predictive model based on machine learning to support the stakeholders of judicial processes in planning their action more efficiently.
Naturally, internal stakeholders, public administrators and officials, may constitute a significant catalyst that can ease the entering process. In order for these actors to be able to essentially contribute to the digitalisation process, they need to be supported and potentially re-educated (Matt et al., 2015). Among others, these users can contribute to the generation of big open legal data and the related business process reengineering processes (Grover, et al., 1995). The special know-how on the user's side typically includes ontology mapping, metadata scheme development and the creation of legal document templates and can be used to meet local customisation demands. More advanced skills might become necessary later in the law-making process such as drafting techniques and homogenous handling of legal texts.
Interoperability is a critical feature for efficient systemic integration, especially within the European Interoperability Framework (Kouroubali & Katehakis, 2019). This is also true for legal interoperability, which is a central component when developing an ecosystem of interconnected legal documents within a certain legal order (a national state) or across different ones (the European Union, for instance). The 2017 Tallinn declaration (2017) called upon 'to fully integrate digital considerations into existing and future policy and regulatory initiatives', thus recognising the need for digital-ready legislation. Such legislation needs to be interoperable. Legal interoperability is about ensuring that organisations operating under different legal frameworks, policies and strategies are able to work together. A specific recommendation for legal interoperability included in the new EIF proposes public administrations to '[e]nsure that legislation is screened by means of "interoperability checks" to identify any barriers to interoperability' (European Commission, 2017).
Interoperability resources can be reused through different types of capacity building activities. The implementation of an educational framework to promote the necessary capacity building on interoperability has been conducted by the Interoperable Europe Academy, formerly known as the Interoperability Academy, of the ISA 2 programme and can be promoted by national schools of government in a structured manner.
This article discusses a knowledge management experiment in Greece that has been conducted through the establishment of a legal interoperability laboratory at the National School of Public Administration and Local Government, known as ESDDA (or the 'School'), in Athens. ESDDA is part of the National Centre of Public Administration and Local Government (EKDDA or 'the Centre'). At the core of this laboratory lies the Legislation Editing Open Software (LEOS) system aiming at creating the conditions for establishing a legal knowledge management base that can be utilised by various degrees of the national administration. Hence, the article describes the empirical study on the training of LEOS, an operations support system (OSS) tool for legislative work that can be directly linked with advances in the broader knowledge economy discussion, as defined by Powel and Snellman (2004). It outlines the international educational reference framework that made the interoperability laboratory possible while highlighting the preparatory actions for establishing it.
ESDDA's approach is considered innovative because it directly transformed scientific knowledge and practical field expertise into a pilot course for public administrators. In particular, the focus is placed on the training aspect for the trainees to develop the necessary digital skills to operate a complex piece of (legal knowledge management) software like LEOS. The technical and configuration parameters of LEOS are presented, and the utilisation of the system is linked to the necessity to efficiently manage and broadly disseminate legal information. In this regard, the research question this contribution attempts to tackle is related with the optimal laboratory setup and the mix of training parameters to efficiently facilitate the necessary knowledge transfer. For this, the evaluation results of both the laboratory and the training process was analysed and discussed.
Following the 'Introduction', the article has the following structure: the second section highlights the national and international educational framework that paved the ground for the interoperability lab and the methodology for creating it. The LEOS system and the basic considerations for the making of a knowledge management tool for legal information at the national scale are shown in the third section, followed by the presentation of the laboratory and the related course (in the fourth section). The fifth section discusses the empirical results, before moving on to the concluding thoughts (sixth section).

Digital Skills for the Public Administration
People are not born with digital skills. They need to be taught. The Digital Education Action Plan (2021-2027) is a European Commission (EC) framework to promote strategies and projects to improve the level of digital skills in Europe. The plan's strategic priorities are twofold: to foster the development of a high-performing digital education ecosystem and to enhance the digital skills and competences for the digital transformation. In addition, an Open Educational Framework (OpenEdu) was designed to promote openness in education (Inamorato et al., 2016). When it comes to the public sector workspaces, a digital shift is apparent. Consequently, the capacity of public administrators and officials needs to be adjusted. In other words, a digitally skilled workforce needs to be in place to make digital government happen (Chinn et al., 2020). For the digital workers to perform, optimal working environments as well as the proper skills are necessary to achieve moving from e-government to a truly digital government (OECD Framework, 2021).
ICT competences are demanded, while the general trend goes towards markets that ask for specific practical abilities rather than specific degrees. It is therefore important to develop and certify digital abilities that determine an individual's professional development within an evolving technological landscape (Van Laar et al., 2017). Such abilities and competencies are part of an administrator's professional profile. The European Union and other organisations such as the OECD promote such frameworks that include core digital skills and competencies (DigComp, EntreComp, e-Competences Framework, EIFISC, etc.). These efforts are supported by a wide range of studies that analytically screen the working environment to determine the necessary (digital) profiles (for the case of interoperability, see for instance, European Commission, 2021). As needs, trends and habits in the workspace are evolving rapidly, so shift the professional profiles. Frequent re-evaluations are therefore necessary to ensure an optimal alignment.
In Greece, the responsible authority for promoting digital skills is EKDDA. Thus, it is responsible for building up the capacities for efficiently implementing the national digital government policy. This is achieved by developing courses that lead to appropriately skilled and effective public servants that originate from both the regional and the central government level. The Centre adopted the Open Educational Framework that, among others, utilises open educational resources to strengthen the digital transformation of the public sector (Mikroyannidis & Papastilianou, 2021).
Its efforts take the InCiSE, 2019 ranking into consideration according to which the Greek public administration is underperforming in sub-sectors such as employees' skills, the production of laws and regulations, the digitalisation, public policy making and transparency (InCiSE, 2019). In effect, building of more specialised professional profiles seems to be necessary for the public employees that include basic competences such as 'good regulation of public policies' and 'legislativetechnical training principles for regulatory acts'. Moreover, the Centre specifically promotes the notion and patterns of interoperability (also legal interoperability) as the enabling conditions for digital transformation across levels of government (Papastilianou et al., 2020). For this reason, distinct actions have been put in place to create the conditions for modern law-making in the Greek Executive State that incorporates better regulation and good governance principles. On interoperability in the Greek Executive State, one may further consult Fitsilis and Kalogirou (2021). In particular, following evaluation of the aforementioned InCiSE, 2019 report, ESDDA's curriculum was reformed to promote methods and profiles that account for an improved national regulation system.
Based on the above framework as well as pre-existing courses on good legislation, public consultation and law-making, the School (ESDDA) designed and offered an interoperability laboratory and the related course to advance digital law-making skills within the public administration. This duo, the course and the laboratory, also to be collectively called 'workshop' or simply 'laboratory', was named 'Laboratory for legislative drafting and the processing of regulatory texts' and constitutes the test bed for conducting a wider knowledge management experiment for legal information and, more precisely, big open legal data (BOLD) (Wass et al., 2016). For designing it, a mixed-method was opted that uses open collaboration schemes, cocreation approaches and advanced ICT tools (Valverde-Berrocoso et al., 2020).
The approach of choice simulates the law-making and the consultation process through a project and problem-based learning method to extract specific results in all building steps (De Graaff et al., 2007;Chen & Yang, 2019). The participative dimension was strengthened via e-learning techniques that included open tools, open standards and mechanisms. An adapted Salmon e-learning methodology was used to familiarise the students with the co-formation of legal rules through their active participation and simulation of the entire process (Zalavra & Papanikolaou, 2022). During the e-learning process, the students adopted specific roles in the law-making processes and provided feedback (Mateescu et al., 2021). The design and delivery process passed through the following six-step process: instructional design, community building, collaborate learning, law co-formulation, simulation in real environment and optimisation and course evaluation.

LEOS and Its Use as a Legal Knowledge Management Tool
The discussed laboratory aims at the legal and technical processing of documents using Legislation Editing Open Software or LEOS, which is an open-source software tool implemented under the European ISA 2 programme (Action 2016.38, Legislation Interoperability Tools [legit]) (European Commission, n.d.). Its open nature, the possibility of wide configuration and its special features make it an interesting case for the Greek public administration. LEOS is developed as an open-source tool to support the law-making process and facilitate online cooperation for drafting regulatory documents and making them available in an XML format based on the AKN standard. The graphical user interface (GUI) and the overall philosophy of the system limits the user's engagement to the purely legal part. This is achieved by making custom document templates available to the users that correspond to different categories of legal text (directive, regulation, decisions and others).
Using the 'drag and drop' function, users can create chapters, articles and other parts of the legal document. The web-based WYSIWYG approach allows users to process the individual details of the document. This ensures that any content written by users follows predefined rules, thus avoiding systematic errors and discontinuities, e.g. in structure and numbering. LEOS provides complementary services to facilitate online cooperation. By using the version control function, the tool allows users to compare previous revisions and modifications of a document. In addition, it offers the ability to simultaneously edit a document with other users and export it to HTML and PDF formats while altering its metadata. The fundamental elements of the system are designed to enable: cooperation (in terms of collaborative legislation), comments and suggestions, rich-text elements (such as images, tables and mathematical formulas), document import (for instance from national printing offices), version control and structure customisation (through the use of templates that introduce restrictions on the structure and content of the document).
LEOS, as a general law-making tool, needs to cope with the diversity of the legal orders within (but also outside) the European Union and their respective legislative traditions. It is therefore designed in a modular way that supports, among others, the use of different languages. It is also crucial for the system to be able to adapt and interact with the various pre-installed systems in the target organisation such as those relating to user management, digital signatures, email servers and database systems. In order to meet these needs, its design architecture is based on the utilisation of reusable and tailored blocks within three layers: the web layer, the service layer and the back-end layer. The web layer is the front-end that contains the user interface (UI) with the system. The interconnection with external systems and services (integration services) such as notification systems, document import and the conversion of the document into various forms takes place at the service layer. The fixed repository (database) and content management services are located in the back-end. One of the important features of LEOS is certainly the ability of the user to formulate templates for drafting legislative and other regulatory texts. The creation and formatting of the templates take place outside the LEOS GUI and are, therefore, not accessible to unauthorised users. Fitsilis et al. (2022) offer a more analytic description of the LEOS system architecture. The tool's interoperability was proven by successfully linking it to external sources of legal information, i.e. written parliamentary questions (Leventis et al., 2021). Hence, it is considered possible to economically integrate LEOS in flexible technical platforms, for instance, through a software-as-a-service model for parliamentary organisations (Leventis, 2022).
Particularly important is the fact that the development of LEOS is supported by the European Commission and the Directorate-General for Informatics (DG DIGIT). The application is available under the public software licence of the European Union, EUPL. In recent years, a community of users and supporters has evolved around the tool, who contributes to its development by writing code to add new features and by developing their own use cases. In the first stage of its development, the tool supports only the European Commission legal texts but provides the possibility of extending towards other legislative procedures. This ability has been exploited for the making of ESSDA's interoperability lab that is going to be outlined in the next section (section 'Laboratory and Course Description').
The tool was considered appropriate for achieving the laboratory's objectives for two main reasons. First, it is because by definition (see its aforementioned founding parameters and operative principles), it constitutes an interoperability solution that supports collaborative work in the regulatory domain. Second, it is because by establishing it, the administration gradually creates a homogenous base of legal documents. Once operable across all administrative levels and powers, legislative, executive and judiciary, it is possible to create an 'ecosystem of interconnected legal documents' that essentially constitutes a full legal knowledge management base. Using LEOS, users can navigate within the ecosystem generating new and 'consuming' existing legal information.

Laboratory and Course Description
The course aimed at enhancing the understanding for drafting and handling of structured legal documents as well as showcasing the gains from the management of the legal information encapsulated therein. Additional objectives included the broad familiarisation, theoretical and practical, with advanced legal informatics tools and concepts. It was exactly this exchange that provided the students with the necessary skills to thrive as digital transformation experts in the public administration. The actual workshop took place between May and June 2020 and included a state-of-theart computer lab running the LEOS application. However, the pandemic disrupted the original course plans that eventually migrated fully online. Hence, the trainer was authorised to locally establish the necessary ICT infrastructure including installation and configuration of the necessary software tools.
Lecturing was performed via Webex video conferencing app, while the relevant legal informatics tools and processes were demonstrated online. Ten students from the 26th ESDDA class participated in this interoperability lab, from which 30% had a computer science and engineering background. The rest had studied law and international studies (30%) and economics (40%). The student's background was identified early on as a potential negative factor to assimilate the technical context provided through the laboratory. Therefore, the use of technical language during lecturing was avoided to the extent possible, and a 'digital backpack' was made available that included the lab's manual, online presentations and supplementary material (scientific articles, reports and software).
The online course consisted of three teaching modules of four hours each: legal informatics theory and standards, legal informatics in practice and methodology of legislative drafting. The first module introduced the students into legal informatics by presenting legal documents standards and ontologies such as Akoma Ntoso (AKN), European Legal Identifier (ELI) and Data Catalogue Vocabulary (DCAT). The second module displayed advanced modelling and analysis methods to legal documents. The texts were broken down to their ontological and semantic elements, which are then matched to the attributes of existing standards and vocabularies. This step was critical for the students to understand the added value of LEOS, which was dissected with an emphasis on its architecture, basic configuration and limitations. The third module was dedicated to showcase LEOS through the study of a use case (a regulatory example). A legal document was transferred into the LEOS AKN-compatible XML format, and metadata were added. The course and the results of the legal transformation process were evaluated and conclusions were drawn.
A functional instance of LEOS (R3.0.0) was used. Technical assistance was provided by the LEOS Team at the Directorate-General for Informatics (DG DIGIT) of the European Commission and the Hellenic OCR Team, https:// helle nicOC Rteam. gr, a scientific initiative that managed the technical complexity for installing, configuring and parameterising the open-source tool. Other open-source software such as a text editor and an ontology development tool were also used (Notepad + + (with the XML tools plug-in) and Protégé (R4.3), respectively). LEOS functionality was demonstrated using a concise legal document, a presidential decree (PD) (PD 16/2020, Government Gazette [FEK] A'29), that entails basic structural elements to be modelled in AKN such as title, references, articles, date and signatures. As a direct interaction of the student with the tool was not possible at the time, a detailed mapping of the steps required to fully transfer the above legal document into the LEOS environment and export it as open legal data was attempted (walkthrough).
Empirical evidence was utilised to respond to the research question. This pilot use case of LEOS within a laboratory environment and its use as a knowledge management tool were evaluated in detail by workshop participants. As mentioned, the LEOS laboratory was attended by ten students. The aggregated results from all eight ESSDA laboratories with 102 students (response rate 86.2%) were used as a benchmark for comparison purposes. The evaluation had four dimensions: • Type: qualitative (regular expressions allowed), quantitative (Likert scale, value: L) • Means: online polling, questionnaire • Timing: during training, ex-post • Level: tools, laboratory The following section discusses the empirical data obtained via online polling during the course and questionnaires that were circulated and collected after its conclusion. Both the tool (LEOS) and the laboratory as a whole were evaluated using qualitative and quantitative data. The questions posed and the raw data obtained are shown in the Appendix.

Empirical Results
In general, evaluation covered organisational and training aspects. A 10-point Likert scale was used. Concerning organisation, five laboratory parameters are covered: course organisation, course content, content coverage, quality of training material and examination type. Here, the interoperability lab (mean average L = 8; σ = 0.38) systematically underperforms by ~ 9% in the Likert scale compared to the rest of the laboratories (mean average L = 8.9; σ = 0.22). The limited course time and the demonstration mode instead of a, surely preferable, hands-on computer lab might have played a role in the ratings. Training was evaluated through the following parameters: presentation of the topic, content coverage, coherence/consistency, cooperation and means/ways of teaching. Hence, it is more linked to the trainer's performance. Similar results with the above were obtained; i.e. the Likert values for the interoperability lab (mean average L = 8.5 / σ = 0.40) were consistently lower compared to the rest (mean average L = 9.3 / σ = 0.15). It is noted that this stable difference in grading along several parameters and course aspects is sometimes not consistent with the qualitative evaluations (as per regular expressions). This discrepancy could be attributed to structural circumstances (this was the only computer lab in an otherwise theory-dominated institution) and should be further investigated in future implementations of the interoperability lab.
Furthermore, a series of aspects around the use of LEOS were captured via online polling. A 4-point Likert scale (1, very good/high; 2, good; 3, average; 4, problematic) was opted here as a high granularity level was considered not fit for purpose. These covered, among others, the issues of the tool's functionality, user-friendliness and technical specifications. All students acknowledge the tool's functionality, i.e. 100% described LEOS as 'good' and 'very good'. This is an encouraging result for developing further this digital solution as a web-based cooperative law-making solution for the Hellenic legal order. Similarly satisfying results were obtained when it comes to user experience: 90% perceived it as 'good'. This result is likely to be impacted by the indirect interaction with the tool, so in reality the parameter could be valued significantly higher. The technical specifications were highly evaluated too (80% of the trainees think they are 'good'), which could mean that they were plainly explained by the tutor. Nevertheless, given the low participation of technical experts, this assessment should be taken with a grain of salt. Of high importance, however, is the opinion of the 78% that LEOS may provide 'high' added value to the law-making process. No clear stance is taken in the matter of the tool's configuration, which is clearly received with mixed feelings (specialised knowledge and prior experience in ICT are necessary to install and configure LEOS).
Summarising, the evaluation encourages further investigations on LEOS as an authoring tool and supports further iterations of the interoperability lab for the next ESSDA classes. While screening the relevant literature, one realises the scarcity of similar efforts to centrally train public administrators on emerging legal informatics tools. Since there can be no comparison with alternative training setups, the current assessment can only be used as a point of reference for future national or international efforts.

Concluding Remarks
The setting up of an interoperability laboratory at the National School of Public Administration and Local Government in Athens, Greece, was described. At the core of the laboratory there was LEOS, an open-source authoring tool for regulatory texts. On this basis, a course with ten participants was successfully organised and conducted online. The exercise also turned out to be a feasibility study for utilising the tool within the public administration. The conditions for the training course were approximated by studying several evaluation parameters. In addition, the use of LEOS for the creation and management of a knowledge base for legal texts was investigated.
The analysis of the empirical results offered encouraging results regarding the tool's functionality and the related user experience. As a result, one may conclude that it can offer substantial added value to the wider regulatory process. More research is still necessary to first reproduce these findings and then to attempt the move from the pilot to the production phase. Several worrying issues were recorded too. These mostly regarded the installation and configuration issues, the tool's opensource nature and its midline position in the development pipeline, each of which may constitute a high entry barrier. Some of these issues are currently addressed by the European entities behind LEOS development, which suggests a close collaboration between the Greek and the EU administration to minimise investment and shorten the necessary development and customisation times.
This experiment also favoured the study of a proof-of-concept for a consolidated knowledge base of legal texts and, thus, the subsequent use of LEOS as a knowledge management tool. As legal texts are homogenised while entering LEOS, they immediately constitute machine-consumable data. Whole new degrees of managing this data wealth are opening up starting from generating ecosystems of legal documents within national borders. Due to intrinsic interoperability features of the LEOS system and the underlying standardised legal document schemes and ontologies, an interconnected legal space can be constructed for the unified provision of legal information and services across different legal orders (for instance, across the EU).
The feasibility of the interoperability concept within the Greek governance sphere is the main lesson learned from the design and implementation of the laboratory. Further research by Leventis et al. (2021), who repurposed the tool for processing written parliamentary questions (a specific type of legal documents), and within the ManyLaws project, https:// www. manyl aws. eu/, validated the concept's feasibility. Additional lessons learned during the implementation of the interoperability laboratory point at the necessity to intensify work with (linked) open data and support the development of linked interoperability labs and extension courses. For this reason, a link with the Interoperable Europe Academy (2021) could promote exchange of good practices (see, in this regard, Kalogirou et al. (2020)). A future implementation of the interoperability laboratory needs to be closely linked with the on-going development of the National Portal for the Codification and Reform of Greek Legislation .
The results from the evaluation together with some of the lessons learned from training implementation create fertile ground to confidently answer the original research question. In regard to the part related to the optimal laboratory setup, a physical implementation appears preferable compared to the virtual one. Technical assistance support must be provided, e.g. on the spot or via dedicated help desk, for the trainees to be able to devote themselves in the learning process rather than dealing with technicalities. A cloud-based solution was made possible at a later time and could even prove more advantageous, thus further limiting the necessary resources for training implementation. Other parameters to further optimise knowledge transfer may include the building of a trainer pool; a longer course period; the development of detailed user manuals in the native language (Fitsilis & Makropoulou, 2023); and permanent cloud-based access to a software instance by trainees to familiarise themselves with the tool.