A systematic review of the literature in Europe relating to clinical legal education

As my PhD research is European focused, looking at knowledge, skills and attributes development in live client clinics, I wanted to find all the European literature relating to clinical legal education. The aim of this research was to find all of the European literature surrounding clinical legal education available to me, to explore the kind of research published and to identify any gaps in knowledge. With an explosion of literature within the field, and more research undertaken every year, finding the literature which related to my research was challenging. To help aid this work I embarked on a systematic review, building on work by Tribe Mkwebu, [1] systematically searching for peer reviewed articles. This research was initially presented at the European Network of Clinical Legal Education ’ s Spring Workshop, 2015, Northumbria University.  This article highlights the journey through this literature. Firstly, it explains what a systematic review is and how it can be used within mixed methods research. It then goes on to outline the methodology used and the number of articles sourced, excluded and synthesised. The analysis shows the amount of papers published before 2015 and their basic content. Finally, I discuss my reflection on the systematic review, what I thought went well and what didn’t, explaining how it was received at the Workshop.

Reviewed Article -Research and Impact 83 paper to be used more as a guide of conducting systematic reviews within our field and how it can be used to further our collective research agenda.
What is a systematic review?
A systematic review, as Khan et al suggest, is, 'a research article that identifies relevant studies, appraises their quality and summarises their results using scientific methodology.' 6 The amount of research conducted and published into CLE has increased greatly over the years and the possibility of reading it all is not likely, if not impossible. A systematic review can combine research questions and key words in order to perform a rigorous search for literature, narrowing search results.
There are key phases to a systematic review, formalised through the Cochrane Collaborative over 20 years ago, which were followed during this review. The key phases are: The advantages of conducting a systematic review are quite strong, in a methodological sense. For example, it allows a comparison of research papers available in order to assess consistency of the research. Due to its robust nature and its methodological structure results are often more reliable or accurate. 8 This is due to its scientific nature. Its purpose is to aid a particular question or hypothesis and test it. With my particular question in mind a systematic review appeared a better fit. To attempt to find all of the peer-reviewed literature regarding CLE in Europe, the scientific methodological way of searching, using key words and wide searches, meant that I was more likely to answer my research question, as opposed to using a narrative literature review. As I had searched in such a comprehensive manner, searching all of the databases I did and following the Cochrane Collaborative model, There are limitations to this methodology in order to locate relevant literature, which will be discussed in more detail below. It is a particularly time-consuming process, which can take weeks to complete. However, the rigour and comprehensive nature of this research methodology ensures that essentially all the articles in a certain area on the databases are 85 found. The comprehensive nature occurs because I am finding all of the relevant peerreviewed articles published to answer my research question. When I say a systematic review is rigorous Piper suggests that: 'When faced with any question, being able to conduct a robust systematic review of the literature is an important skill for any researcher to develop; allowing identification of the current literature, its limitations, quality and potential.' 9 This rigour applies to appraising the research as well as finding it. Finding the research and following the set structure means that all literature identified should be in the research area.
At the appraisal stage articles are excluded which are not deemed to eligible for inclusion and those left are submitted to a weight of evidence test. This test seeks to assess the quality of the research that has been found from the systematic review, but is important to note that papers are not necessarily excluded due to appraisal. 10 Rather, sufficient weight should be attached to them in light of the appraisal.
Thus, this systematic review is not only identifying the literature already available in Europe, but also its strength, reliability and relevance to the research question. This is why it was chosen for this particular research.

86
A narrative literature review differs greatly from a systematic review, and is considered the more traditional approach. They can be more critical, assessing theories, putting the research into context and discussing the background of the research in more depth. Articles are selected by the author, based on his opinions. There are advantages to doing a traditional literature review. For example, they can go into a deeper discussion about the research and its theory, following a trail of different authors, their academic discussions and how their research has developed. It can be argued that this provides a richer literature review, discussing certain elements of research in great depth. Other articles and materials for the review can be found in footnotes and bibliographies, one piece of work providing for another.
However, narrative reviews have been criticised by academics. Garg, et al highlight an issue that, 'It is sometimes uncertain whether the author of a narrative review selectively cited reports that reinforced his or her preconceived ideas or promoted specific views of a topic.' 11 Thus, due to an author choosing what is included in their review, and having the freedom to exclude materials which may not reinforce their opinion or theory, we may not be getting the entire picture from a narrative review. This is the issue of bias. We only see what the reviewer would like us to see, potentially reducing the reliability of the review. Reviewers, after all, are experts in their field, with opinions about the publications which exist; it can be hard to separate that opinion from a literature review. I am not enforcing that every literature review must be systematic and narrative reviews are always unreliable, but a good, methodologically sound, systematic review eliminates the temptation to exclude material which otherwise should be brought to the attention of the reader.
Using a systematic review was the most appropriate literature review for my research question. A narrative review would not have produced the same results and I would have missed many articles if I had used this alternative.

Using qualitative studies in systematic reviews
Systematic reviews, traditionally, contain only quantitative studies within the evaluations of their materials. This is due to their scientific nature, and quantitative studies viewed as supporting a hypothesis or research question more appropriately. However, due to the total amount of papers sourced in this systematic review being conceptual and qualitative in nature it is necessary to discuss the use of these kinds of papers. As Petticrew states, 'Qualitative research can identify the range and nature of impacts of interventions and can give sense of whether they are rare or common. It can identify unintended, unanticipated impacts.' 12 He notes that systematic reviews containing qualitative research are becoming more common, and the impact that they can have should not be ignored. Qualitative studies can help us to answer research questions which may not be answered through quantitative studies, and including them in a systematic review may create a more reliable result to the research question.
Petticrew maintains his argument with Roberts, advancing that, 'There is clearly enormous scope for improving the means of accumulating the knowledge gained through qualitative studies.' 13 Thus, quantitative studies can help to improve the knowledge that is gain through qualitative research, providing more depth to the answer of a research question. The NHS Centre for Reviews and Dissemination now provide, in their guidance of undertaking systematic reviews 14 , the advantages that using qualitative data may provide. Some academics have warmly welcomed this, agreeing that using this kind of data can add a new perspective on systematic reviews, one that quantitative only studies may lack. However, there are still some issues with using this kind of data in a systematic review. Dixon-Woods and Fitzpatrick outline them as: 1) Rigour is important when searching for a systematic review. Searching for qualitative studies can be 'frustrating' or difficult 2) A suitable way to appraise qualitative studies has not yet been agreed. The models for quantitative studies are not appropriate 3) How do we synthesise and conduct secondary-summary with qualitative data? 15 Whilst these are issues for a qualitative systematic review, they are not complete barriers to conducting one. I appreciate that appraising the studies does not have a set methodology, but I believe that if you are clear with your appraising methodology this should not be a huge issue.

89
This systematic review contains only qualitative research, forcing me to exclude the metaanalysis stage of a traditional systematic review. Even though there is no quantitative data to extract and synthesise from the research, there is still a strong opportunity to analyse qualitative data using this method.
Petticrew and Roberts highlight that systematic reviews are a great method that 'can be used to summarize, appraise, and communicate the results and implications of otherwise unmanageable quantities of research.' 16 However, they add to this by stating that systematic reviews can lack an actual substantive discussion of the literature, and the 'communication', element should be greater if systematic reviews are to be 'really useful '. 17 This communication element is more prominent in a traditional literature review approach. I hope that the increasing use of qualitative studies in systematic reviews will result in greater communication of the information available for particular research questions.
As the Cochrane Collaborative follows the key phases, outlined above, I had to modify some phases slightly, particularly the synthesis stage, in order to fit my methodology and the sole use of qualitative research.

Aims of this systematic review
-To find all of the peer-reviewed and published research available regarding CLE in

Europe
-Explore what research has been published in this area (area concerning both research and regional) -Analyse the research, looking at whether it is theoretical or empirical, and the reliability attached to it -To identify any gaps in knowledge or further research that can be conducted.
Looking at the aims of my systematic review, they were partially met. I did find all of the peer-reviewed and published research available regarding CLE in Europe. I did explore and analyse the research. However, I do not think I can state that I found all of the gaps in knowledge or can definitely say what further research needs to be conducted in this area.
This will be discussed in more depth in the section discussing the limitations of this systematic review.

What research has been published in Europe regarding clinical legal education and what is missing?
Formulating my research question was not a particularly difficult task. As I wanted to know what research had already been done in Europe generally it was easy to express in a question.
I was gathering any articles relating to CLE from the continent, and not narrowing it down to specific CLE issues. Thus, I wanted to gather articles that related to establishing a clinic, sustaining it, teaching methods, social justice aspects, assessment or any other research topics

Mapping and scoping
The first phase of this review was to design a plan to conduct the systematic review. I already knew that there are more articles produced in the UK than there are from Continental Europe. However, I knew that some countries had published research and engaged with CLE more widely than others. Thus, there were some countries highlighted for specific searches with a general European search to catch anything that may be missed.
The key words were developed from the research question. 'Clinical legal education' was the main search term as this is the most relevant part of the research question. The remaining key words were the countries I had identified as more likely to have published in this area, and then a general Europe search.
The 'sweeping searches' used more general terms that were wider than the research question, to ensure I had captured all of the materials available to me. The results of these searches have not been included in the final Prisma diagram, as they did not form the main part of the search. They were used as a final check, and did not provide me with any further articles (see appendix 1).
I decided that I would not have a time limit, or a start date, to my searches. I wanted to find all of the articles available to me, regardless of when they were published. This worked quite well, as the modern wave of CLE started in the 60's, mainly in the US, Australia, Canada and Reviewed Article -Research and Impact 92 the UK. 18 As Continental Europe mostly did not begin engaging with CLE until later, I knew I wouldn't have to go too far back. Thus, having a start date for my searches did not really make too much of a difference to narrowing down my searches for Europe.
Comprehensive search: how I searched for the materials

Choosing Databases
As this area covers law and education, I choose databases which I knew would provide either these areas, or predominately providing one of them.
-Heinonline is a database that covers both these disciplines, with many clinical focused journals subscribed to, rather than just educational. Consequently, most of the articles included in the full text analysis were from this database.
-Westlaw is a database that primarily archives legal articles. However, they do bring results up for the International Journal of Clinical Legal Education (IJCLE). Whilst they are not actually subscribed to on this database, it is an open access journal, with all of the archives available online. Thus, the results from these hits were recorded from this database and accessed via the IJCLE website.
-LexisNexis is another legal based database. It was included in the database selection to cover all of the main legal databases that are commonly used in legal research.
-Lawtel is another legal database, but one that does not commonly archive legal education articles. Again, it was to ensure that the main legal databases had been

93
-SAGE is a socio-legal database, and sometimes legal education is included within socio-legal studies. This database was used to search a much broader variety of articles, rather than strict legal databases.

-
The Social Science Research Network, again, is a much more boarder database, that does archive some legal education articles. It was chosen in order to conduct another broader scoping look at the articles that are available.

Searching
In order to find materials on the databases I used the Boolean operators, to narrow down my

Clinical legal education AND Poland Clinical legal education AND Germany Clinical legal education AND Croatia Clinical legal education AND UK Clinical legal education AND Russia Clinical legal education AND Ireland Clinical legal education AND Czech Clinical legal education AND France Clinical legal education AND Northern Europe Clinical legal education AND Europe
Additional 'sweeping' searches for LexisNexis and Westlaw:

Language (Any articles not in English)
Databases not subscribed to by my institution Geographic (any research not produced or conducted within Europe)

 Title
Many articles were included from their title. It was easy to see articles that were highly relevant to the search as they normally stated the country and kind of CLE it is discussing.
For example, it was easy to see from the title, 'The Next Step Most articles results could be excluded by title. They were excluded if it was clear that they didn't fit into the European criteria and were research produced from outside this continent. A total of 44,171 articles were excluded due to title.
Next, articles were excluded if it was clear they were to do with medical education and not legal education. For some databases the majority of articles produced were for medicine and not law. It was very easy to identify which articles were medical and would be excluded by title instantly.

 Abstract
Occasionally it was necessary to read an article's abstract in order to determine whether it should be included in the systematic review. Abstracts read had to illustrate that the article was research conducted within Europe and relating to CLE. This was normally fairly easy to identify. If it was not, then the contents page or introduction were read to determine inclusion. This did not, however, occur very often and it was not deemed necessary to make a record of how many articles were included from contents page, etc. They were included under the abstract criteria heading. Altogether 120 articles were excluded by abstract.
The reasons for exclusion by abstract varied. The majority (approximately 100 articles) of abstracts excluded were due to the research not having been conducted in Europe.
Often, searches would produce articles that mentioned a country in the footnotes or briefly within the main body. These articles obviously would not answer my research question and were not relevant to the systematic review.
Articles were also excluded if the abstract did not mention CLE at all, meaning that the research didn't concern this kind of education, but legal education more generally.
 Articles that had to be excluded Some articles had to be excluded for various reasons. If the article's full text was not available or not subscribed to then it was excluded. If there was no possibility of gaining the full text then it could not be included in the later stages of the systematic review, the synthesis. A total of 29 articles were excluded for unavailability.
A small amount of articles were excluded as they were written in a language other than English. There was no means available to translate these articles at the time of the search. There were only 2 articles excluded due to language.
Lastly, there were articles excluded as duplicates. These were articles that had already been included to be read in full text and thus were not needed again. The articles excluded during a search were those that were relevant to the country searched. There were a great number of articles produced for medicine and health, and they were not recorded again as duplicates as they were not relevant to my search. Only those to be read in full text were excluded as a duplicate. A total of 34 articles were excluded as a duplicate.

 Full text
Only 3 articles were excluded after reading full text. These reasons were as follows: Reviewed Article -Research and Impact 98 -One article was not research in Spain. It was actually research from a US university that had the same name as a Spanish city. Once realised this was not European research it was excluded.
-One article was excluded from Germany. When reading this article it came to light that it was only about legal education in Germany and did not mention clinical legal education at all. As it did not mention any clinics, and other articles provided a comprehensive background to legal education in Germany, I felt it was not necessary to include this article in the final results.
-An article was excluded after it was realised that it did not really concern CLE, but rather developing an educational system for preparing students for international practice. This was not relevant to this systematic review and was excluded accordingly.
As can be seen, there were not many articles excluded during the full text stage of the review.
I believe this is because my screening worked so well, and articles that would fit the criteria were chosen carefully.

Recording the articles
Once I had identified articles I wanted to read in full text, I recorded them on a I only recorded the information of articles I chose to read in full text. Articles that were excluded prior to this were not recorded in more than a numerical figure, as it would have not served me any purpose.

Findings
The records I identified before screening, for all searches conducted, are as follows: -Heinonline: 23,147 published research, I expected to find more articles than I actually did. This was especially for Poland, Croatia and Ireland. I was not surprised by the results for the UK, as I knew this country would lead the amount of research available to me. The results for Germany were surprising as this is a country which has taken longer than others to engage with CLE. 21 However, I realised that some countries had not produced any research outlets themselves in my results, but rather it was mainly US academics writing about their experiences of CLE in these countries. This is especially the instance for Germany and Russia. Some countries were mixed and some produced all their own research, mixing it comparatively with other countries. This is shown in figures 3 and 4 below, which compares domestic to non domestic articles produced per year.
I think I was expecting to find more articles than I actually did in the final result. However, it can't be forgotten that there were many articles excluded as they weren't available to me.
Had they been, the results would have been different, particularly for the UK.
Below is a Prisma flow diagram 22 , showing the different stages of my systematic review and the inclusion/exclusion numbers. I have already dissected above how many articles were excluded for various reasons and why, and it is not necessary to repeat them here. 21 Tomoszek M, 'The Growth of Legal Clinics in Europe -Faith and Hope, or Evidence and Hard Work?' (2014) 21 International Journal of Clinical Legal Education 93 -this paper highlights, particularly at p.99, how Germany has 'resisted' the CLE trend, favouring the more conventional approach to legal education. 22 The findings included in this Prisma do not include the records for the additional 'sweeping searches'. There were no records used, as shown in the table in appendix 1.   Europe and the difficulties they faced. As these clinics become more established we begin to see a rise in the amount of papers published by domestic authors.
However, due to the limitations of my systematic review, discussed below, there does seem to be very valid reasons for domestic authors not publishing as much peer-reviewed CLE research, especially in the earlier years.
I think these results can also be represented on a graph, comparing papers published by domestic to non-domestic authors, rather than total papers published compared to nondomestic. Representing the results in this way appears like this:

Figure 5 -Graph displaying the difference in numbers of papers published by domestic and nondomestic authors
Looking at this graph I can conclude that in some years, especially in the earlier years, there were more papers published by non-domestic authors than there were domestic. Whilst this graph is similar to the one above, and the same conclusions can be drawn, seeing it presented visually different on this graph displays the occasional contrasts. I found these differences a particularly interesting aspect of the results, prompting me to draw conclusions as to why.
This will be discussed further in the next section, whereby the analysis of the content of the papers can attempt to form a theory to explain this difference.
Synthesising the quantitative elements of the literature Poland (5) Ireland (3) Croatia (2) Czech (2) Norway (1) Netherlands (1) Serbia (1) Romania (2) UK (11) Papers explaining generally legal education and CLE 21 Poland (2) Germany (4) Ireland (2) Italy (1) Czech (2) Russia (3) Netherlands (1) France (1)  By studying this table, we can see where the trends in publications are. It is very common for those working in clinics to write about establishing a clinic and how to run one. However, it was found that these papers were mostly not followed up afterwards, with no papers explaining the development of the clinic or the setbacks. Whilst these papers can be very useful and enlightening to others working in and wanting to establish a clinic, it is also important to discuss if a clinic has failed or any setbacks and difficulties that have been faced.
By sharing this information we allow others to learn from our own experiences and to help build collaborations or forums for advice.
Clinical legal education, and legal education generally, was also found to be discussed widely in the literature. What are the downfalls of legal education, how can clinic help to fill this gap? These were questions widely addressed in these papers. These are very valuable questions, but ones which have been discussed to a great extent already. This is demonstrated more, the further down we move in the table. Papers discussing specific areas of CLE, such as assessment and integration, become more scarce. Furthermore, these papers mostly come from the UK. As highlighted above, it is necessary for us to be sharing experiences. It may have been that I could not find papers relating to these areas from other European countries, or they may have been in another language. However, I feel it important to emphasise, again, the necessity of us sharing research and experience. Perhaps it is time for us to move away from the general discussion of CLE and focusing on the specifics. Now with the knowledge that there are other publications out there, and a mass of grey literature, I have the means to move forward with this systematic review and look at more unconventional research outlets. Since this work was undertaken I have completed a further systematic review, which forms part of my PhD thesis. I found that that systematic review was much more advanced than this. For example, the duplicates recorded in this systematic review were only duplicates which of the full text articles already selected to be read. In my PhD systematic review duplicates were recorded as any article which had already appeared in my search results. This way I could assess more easily when my searches were becoming saturated, and it gave a more realistic account articles appearing. I hope this work provides guidance for others wishing to undertake similar research and that any hindsight I may have had will inform others.

Conclusions
Whilst there have been some limitations to this study, some conclusions may still be drawn from the methodological results, as well as the review of the literary content of the paper.
From the results it can be seen that Continental Europe is producing more research now, than it previously did. It can also be concluded that there is now more domestic authors publishing research compared to non-domestic. Whilst the limitations discussed above may