Methodological aspects of measuring the effectiveness of the EU participatory mechanisms: the case of the European citizens’ initiative1

This paper proposes a research design which allows for a comprehensive assessment of the effectiveness of the European citizens’ initiative (ECI) – the first EU transnational participatory tool. It provides an analytical framework based on a broader conceptualisation of the notion of “effectiveness” which goes beyond the sole legislative function of the ECI. Consequently, the paper suggests to evaluate the ECI in five different dimensions: as a dialogue and communication enhancing tool, an awareness-raising tool, a deliberative space, a citizen-activating mechanism, as well as an agenda-setting instrument. Operationalisation of the dependent variable through a set of democratic functions that the ECI might perform allows not only to evaluate its capacity to enhance the democratic legitimacy of the EU but also to identify the tool’s limitations and propose ways of improvement. The paper also suggests a qualitative method of testing the hypothesised effects of the ECI.


Methodological aspects of measuring the effectiveness of the EU participatory mechanisms:
the case of the European citizens' initiative 1   In the current political system of the European Union (EU) the Lincolnian function of "government by the people" -which presupposes that executive powers are exercised in response to the articulated preferences of the governed -suffers from relative weaknesses.
In spite of the existence of a wide array of citizens' participation mechanisms (Witkowska 2015), the EU is criticised for its shortage of input-oriented legitimacy (Scharpf 1999), both in its participatory as well as representative dimensions.At the supranational level, the EU lacks the obvious tools of direct democracy such as referenda, and its pre-legislative consultations and impact assessments are thought to be dominated by large corporate interests to the detriment of pressure groups and NGOs (Borońska-Hryniewiecka 2015).The representative link exercised by the European Parliament is not satisfactory 2 and the involvement of national parliaments in the EU affairs is limited to the non-binding exchange of views with the Commission within the so called "political dialogue" and controlling the observance of the principle of subsidiarity (Borońska-Hryniewiecka 2017). 1 This paper presents a methodology used by the author and her co-author Elisabeth Monaghan in a research project Building a European demos: democratic legitimacy in the post-Lisbon European Union and its impact on global governance coordinated by the Jean Monnet Centre of Excellence on EU Law and International Relations at the University of Deusto in Bilbao within the EU Lifelong Learning Programme, EAC/S07/12 (2012/C 232/04).The results of the project were published in a collected volume with Palgrave Macmillan (2017). 2 Since the first direct elections of the European Parliament, every following voter turnout has been lower, passing from 62% in 1979 to 43% in 2009.Since the EP elections do not offer the electorate a vivid choice between competing policy programmes as in a standard parliamentary democracy, the EU citizens do not identify with European parties as they do with national counterparts and EP elections are thought to be "second-order".
In this context, the European citizens' initiative (ECI) 3 can be viewed as a unique expression of direct democracy at a supranational level.By granting one million citizens a formal right to propose policy solutions for twenty-eight member states of the EU, it constitutes the first tool of transnational democratic participation in the world (for transnational participatory processes see: Witkowska 2013).Yet, the extent to which its implementation actually contributes to enhancing the Union's democratic legitimacy is not as self-evident as it may seem.While most contributions published before the ECI was launched praised its democratic potential (e.g.Garcia 2012; Conrad 2013), the evaluation of its performance to date presents a rather pessimistic picture (Berg, Thomson 2014).One common denominator of this assessment is that the ECI does not deliver the expected results as a policy-initiating tool4 .That is because its constituting regulation does not impose binding legal obligations upon EU institutions to act, even if prompted to do so by one million citizens.
Yet, in a decision closing her own enquiry into operation of the ECI, the European Ombudsman emphasised that, in terms of the substantive effects of the instrument, the Commission coming forward with a legislative proposal should not be the only measure of success5 .In fact, considering the ECI solely in legislative terms limits an understanding of the range of its potential implications.The latter are important because, despite the obstacles, they reveal surprising democratic potential.Consequently, this contribution builds on the premise that any comprehensive evaluation of the ECI should be based on its broader conceptualisation which would allow to extract its democracy-enhancing aspects from the moment of its conception, through gathering of signatures to prompting the reaction of the European Commission.Along this line, it proposes a research design that allows to measure the effectiveness of the ECI beyond policy-initiating function, as a communication and dialogue enhancing tool, a deliberative space, awareness-rising and citizens activating instrument, as well as agenda--setting mechanism.
The paper first offers an overview of approaches and methods applied to study the ECI in the literature, to later present an analytical framework through which the instrument can be assessed.To this end it provides a broader conceptualisation of the notion 3 Implemented by Regulation 211/2011.ECI might be brought forward to the European Commission by not less than one million citizens representing at least a quarter of EU Member States and should refer to matters or policy areas over which the Commission enjoys legislative powers. of the ECI "effectiveness", followed by its operationalisation, methods of testing and data sources.It finally sketches avenues for further research on the ECI.

The state of the art
The literature on ECI offers a wide array of studies analysing the instrument from various angles.For example, in the special issue of "Perspectives on European Policies and Society" Victor Cuesta-Lopez (2012) analyses the ECI from a formal point of view.By using the method of comparative law, he identifies the best practices regarding organisation of popular initiatives in national contexts and confronts them with the procedures laid down in the ECI regulation.While optimistic about the ECI's design, Lopez predicts that its ultimate effectiveness (i.e.delivering legislative proposals) would depend on the receptiveness of EU institutions.Renate Preukschat (2014) on the other hand, attempts to determine factors behind the success of the ECI through means of comparative qualitative analysis.In doing this, she however assesses only the initial phase of the ECI exercise as illustrated by the number of gathered signatures.Such minimal conceptualisation of the ECI's effectiveness excludes testing the whole range of its actual effects as a tool enhancing the EU democratic legitimacy.
By analysing the ECI through the theoretical lenses of participatory democracy, Elisabeth Monaghan (2012) broadens the scope of the ECI's potential consequences and shows that its introduction at the EU level reflected a shift from perceiving citizen participation solely as necessary for the effectiveness of policy outcomes to viewing it in terms of building a European political community and the empowerment of individual citizens.Taking a similar perspective, Paweł Głogowski and Andreas Maurer (2013) analyse the scope of the ECI's potential hurdles which might limit its anticipated democratic functions.In doing this, they focus especially on the ECI capability as a Europeanisation and citizens involvement enhancing mechanism.Finally, in suggesting agenda for social science research, Maximilian Conrad and Anette Knaut (2014) pay attention to the transformative power of the ECI with regard to EU institutional architecture as well as social relations within the Union.They encourage to explore in what ways the instrument helps build a transnational political agenda, as well as brings about new rules, procedures and modes of communication between citizens and the EU institutions (see also : Conrad et al. 2016).While the above literature presents different perspectives on the ECI, it lacks proposals of a research design which would allow to test the effectiveness of the ECI in enhancing the democratic legitimacy in the EU.Building on the existing studies as well as the theoretical premises of participatory and deliberative democracy, the remainder of this paper presents a research design allowing to assess the effectiveness of the ECI beyond policy-initiating function.

Research design: operationalisation of the dependent variable
For the purpose of further analysis, the concept of "ECI effectiveness" is defined here as the capability of the ECI to enhance the democratic legitimacy of the EU.The underlying argument of this paper is that despite its failure to deliver concrete legislative output at EU level, the ECI should be viewed and assessed in a broader context.It is further hypothesised that the effectiveness of the ECI might lie in performing certain functions (beyond policy--initiation) -and thus exerting a range of transformative effects -of democratic character within the EU system of governance (Borońska-Hryniewiecka, Monaghan 2016).
Having established that the legislative (output-oriented) dimension of the ECI is practically non-existent, the understanding of effectiveness is limited to its input side (Scharpf 1999).Input legitimacy refers to the character of policy process as ensuring adequate representation and participation of the interested parties in all the phases of decision-making (inclusive governance).In the case of the ECI, inclusiveness does not only refer to granting citizens the access to a strategic decision centre by direct, formalised contact with the European Commission.It can be also complemented by other functions performed by this participatory mechanism such as facilitating communication and dialogue, raising the EU awareness, prompting deliberation, activating citizens, and setting agenda for the EU 6 .

Communication and dialogue facilitating tool
This function of the ECI should be analysed in two dimensions: the vertical and horizontal.In the vertical sense, the central mission of the ECI is to connect Brussels institutions and decision-makers with citizens and show them that the EU is accessible 6 An inquiry into the nature of each of these functions might constitute a range of research questions in respective ECI studies, e.g.Does participation in the ECI enhance communication and/ or dialogue between EU institutions and citizens?In what ways the ECI improves the deliberative character of the EU public sphere?Does the ECI perform the agenda-setting function for the EU policy-makers? and responsive.In this sense, the ECI becomes an opportunity to "bring the Union closer to the people" in a way that had not previously existed.It allows for interaction to take place directly between citizens and EU institutions without parliamentary or executive intermediaries.Other participatory structures found within the EU political system have tended to be indirect and informal (Nentwich 1998) or actually in practice aimed at elites rather than ordinary citizens (Magnette 2003).Moreover, anchoring the ECI operational nucleus at the Commission's Secretariat might also improve its image of a distant bureaucracy and give it a better idea of concerns of European citizens.
Yet, evaluating the vertical communicative dimension of ECI requires taking into account two kinds of considerations, namely communication at the technical and political level.At the technical level, considerations relate to operability of the technical infrastructure and guidance that EU institutions offer within the framework of the ECI.In this respect, we should focus on the availability and helpfulness of the Secretariat--General and its staff in the procedure as well as when technical problems arise, for example regarding the installation of software, or the certification procedure.Another aspect to look at is whether the Commission performs an effective intermediary function in communication between the campaigners and supporters of the ECI.This effectively extend the communication potential of the ECI, making it possible to keep the supporters informed about the progress or the outcome of the initiative.
In the horizontal dimension, the ECI can be conceptualised as a tool for building transnational communication networks across Europe.The requirement built into Regulation 211/2011 that an ECI be led by a seven member "citizens' committee" (comprising citizens from seven different member states) entails bringing together different types of individual and collective actors with different organisational and cultural backgrounds.
The requirement of campaigns being genuinely "European" requires coalition-formation based on fostering a common understanding of a policy issue across member state boundaries.To this extent it is consistent with a longer-established tendency on the part of the Commission to favour as societal partners transnational representative associations over multiple nationally-based and organised interests (Greenwood 2011).

Awareness-raising and Europeanisation mechanism
Linked to the communicative and dialogue-facilitating dimension of the ECI, but analytically separable, is a dimension concerning the awareness-raising potential of the ECI.The idea here is that participation in an ECI can serve a cognitive function whereby citizens' knowledge and understanding of the EU is enhanced in the process.
The successive Eurobarometer surveys have suggested that knowledge of the EU institutions is not widespread amongst European populations.Awareness-raising can on the one hand be about increasing citizens' knowledge about the EU political systema form of "learning by doing".On the other hand, awareness-raising can be about the specific citizenship rights of the EU that accrue to citizens.
Bound up in this dimension of the ECI are assumptions about the relationship between knowledge and understanding, and support for and trust in EU institutions and policies.It is argued that, even if citizens do not use this opportunity structure, they should be aware of its existence since it can reinforce a feeling that they are not only objects of Brussels' decisions and policy-making but can be active creators of these policies.The awareness raising element of the ECI might also serve to alleviate the feeling of distrust towards the EU caused by lack of understanding of its functioning.
In other words, the ECI might perform an educative and Europeanising function for citizens and give meaning to the European citizenship.The end of the so-called "permissive consensus" saw an increased understanding of and concern on the part of citizens of the impact of European integration on their lives, with wavering degrees of support for the pace and direction that integration was taking.In the sense that ECI campaign organisers become Europeanisation agents who play an important role of "transmitting Europe" to their grassroots.Apart from increasing their awareness of 'what the EU is all about' they also help operationalising this capital -making people use their knowledge about EU programmes, funds, lobbying networks and so on.This is what Radaelli (2004) calls the European "grammar in use", "the ways of doing things" that change the logic of interaction between the civil society and the EU.

Deliberative space
According to constructivist approach to the EU democratic legitimacy, legitimising takes place through deliberation and inter-subjective interpretation of the surrounding political system (Kownacki 2014).The possibility of an institutionalised discussion, contestation and clash of ideas creates a level of identification with the system which although itself might not always offer preferred (policy) solutions, ensures conditions for expression and articulation of interests.In this vein, in its proposal for an ECI re-gulation the European Commission advertised the initiative as "fostering greater cross--border debate about EU policy issues, by bringing citizens from a range of countries together in supporting one specific issue" (European Commission 2010: p. 2).In the Explanatory Memorandum accompanying its proposal for the Regulation, and in the context of its proposed admissibility check at an early stage, the Commission gave an indication that the process of an ECI was to be valued as a way of contributing to a European public sphere: "a major objective is to promote public debate on European issues, even if an initiative does not finally fall within the framework of the legal powers of the Commission" (European Commission 2010: p. 6) .
Therefore, the ECI can be viewed as a tool prompting deliberation across borders and thus creating a transnational deliberative space on policy issues which are subject to EU competence that are important to citizens.In this regard the ECI and its surrounding processes can be assessed as having the potential to contribute to the formation of a European public sphere in a "Habermasian" view.The deliberative outcomes that occur can take various forms: in the process of securing signatures (which is thought to require a number of interactions and conversations); prior to launch in the process of refining the perceived problem to meet the requirements of the regulation (commission--proofing the initiative); after signature collection in the EP hearings (if the ECI gets to that stage).Ideally, deliberation processes within the framework of the ECI should fulfil Fishkin and Luskin's criteria of deliberative discussion that is to be informed (and thus informative), balanced, conscientious, substantive and comprehensive (2005).

Citizen-activating tool
Additionally, to evaluate the outcomes of the ECI at the system level (i.e.concrete policy proposal, creation of a public sphere or agenda), a researcher should also consider a transformation at the level of an individual.This function of the ECI is consistent with a radical understanding of the relationship between participation and democracy, or in other words, a genuine understanding of participatory democracy.For participatory theorists such as Carole Pateman (1970) the real democratic value of participation was that it promoted human development and made democracy count in people's lives.
In this sense, it should be assumed, that in the process of lending a signature to an ECI campaign the person involved has been transformed from a private individual to an active EU citizen.This "activity" should not, however be only momentary, but remain with an individual in a form of "readiness to participate".ECI could be thus assessed as a tool encouraging individuals to activate their citizens' rights to participate in the democratic processes of the EU such as, for example, elections to the European Parliament.It is hard to expect that such an effect would be widespread.It is assumed that the activating effect of the ECI could be better achieved if the Commission in fact transformed citizens initiatives into legislative proposals.Doing this would entail the EP's involvement in the decision-making process as a co-legislator on the issue.The electoral activation might also be generated during the public hearing organised at the EP, when people get to know MEPs and might realise that they represent their interests.

Agenda-setting tool
Finally, the ECI's effectiveness can also be measured by its agenda-setting function -a feature which is close to but looser than its legislation-initiating dimension.Agenda--setting is understood as a specific stage of the policy-making process even before the policy initiation stage where the legal right of the Commission to make a legislative proposal appears.Instead of focusing on the ECI as a vehicle of prompting the EU policy-making process, this view places value on the ability of citizens to put items on the Commission's -but also other EU actors' such as the EP or the Council's -agenda.
Not every policy idea which is placed on the agenda will end up being taken forward as a concrete proposal, but in the longer term it may serve to draw attention to an issue and signal to policy-makers the support it has amongst the broader citizenry.Each ECI reflects that a group of European citizens felt it was a worthy enough issue to go to the effort of organising a campaign.Even those initiatives which either fail to gain the required number of signatures, or fail the admissibility criteria can nevertheless succeed in placing issues on the agenda.At the end of the day, each submitted initiative has to go through an admissibility check and as such ends up on the Commission's desk.

Data sources and empirical testing
In order to test whether the ECI performs the above-mentioned functions, thus contributing to increasing the input legitimacy of the EU, one needs to engage in a comprehensive qualitative analysis collecting empirical data from different sources depending on the analysed function.And so, in case of communication and dialogue facilitating feature, the researcher would have to select a representative number of the conducted ECI campaigns and track down the quality of communication (both at the technical and political level) based on the feedback of ECI campaigners 7 .To systematise data gathering, a questionnaire covering all the aspects of communication should be developed.The main respondent group would be campaign organisers, not ECI signatories, meaning those individuals directly in contact with EU institutions who could account for their experience.With regard to testing the horizontal communicative power of the ECI, citizens committees should be contacted in order to obtain information about the scope and nature of cross-border networking generated for the purpose of a concrete ECI.Additionally, social media activity surrounding the campaigns should be analysed including Facebook, Twitter or other internet platforms, tracing the scope and nature of the exchange of ideas taking place there, and what kind of side events were organised to foster dialogue and communication on the ECI.
In general, four out of the five analysed functions of the ECI (except agenda-setting) could be tested using questionnaires and semi-structured interviews.It would be up to a researcher to decide whether to create one comprehensive questionnaire covering all aspects of the ECI or to go for separate tools.It could be considered to develop a separate survey to test the citizens activating function with regard to participation in EP elections (possibly inquiring whether signing the ECI affected their thinking about the EU).This survey would most be addressed to a larger target group than the one on communication and dialogue since it would test the hypothesised transformative effect of the ECI among its signatories -not the main organisers (large sample).In the same vein, to test the deliberative function of the ECI, one would have to contact especially those individuals participating in the public hearings organised at the EP as well as those moderating debates through social media or special issue-related events.
The information obtained from the questionnaires could be usefully complemented by the already existing data on perceptions of the ECI of various actors involved in its implementation.The first set of such data includes feedback submitted to the European Ombudsman by ECI campaigners (both individuals and NGOs) within her own-initiative inquiry into the functioning of the mechanism8 .Another includes a report prepared 7 The case selection should obviously include the successful campaigns who managed to gather more than one million signatures as, it would be assumed, they created the opportunity of fulfilling more democratic functions, i.e. the deliberation during the EP hearing.by the ECI Campaign which gathered the real-life experiences of people who developed and ran ECI campaigns as well as the perspective of EU institutions and national authorities (Berg and Thomson 2014).Bearing in mind, that the information obtained from ECI participants (both societal as well as institutional) might present a certain bias, questionnaires could be also cross-checked, where possible, with sources from EU institutions as well as data from official documents and reports.The Commission (European Commission 2015) and the European Parliament 9 have also published their own assessments of the ECI.Interviews with EU officials, representatives of national administrations who have worked with ECIs and technical advisors to the ECI campaigns would constitute important comparative sources of information providing feedback from other side of the spectrum.In gathering information it is important to cover the entire process and different aspects of the ECI -from the formation of a citizens committee through the admittance of the ECI at the EU level, the design of the public hearing until the Commission's formal reaction.

Conclusions
The aim of this paper was to propose a research design that permits to study the effectiveness of EU participatory mechanisms using the example of the European citizens' initiative.To this end, it has suggested an analytical framework based on a broader conceptualisation of the notion of "ECI effectiveness", beyond the legislative function.Consequently, it has proposed to evaluate the ECI in five different dimensions: as a dialogue and communication enhancing tool, an awareness-raising tool, a deliberative space, a citizen-activating mechanism as well as an agenda-setting instrument.In order to test the hypothesised effects of the ECI, this paper outlined methodological options 9 In April 2015 also the European Parliament published its assessment of the implementation of the ECI.The results can be accessed at: http://www.europarl.europa.eu/thinktank/en/document. html?reference=EPRS_IDA(2015) 536343 (1.08.2016).consisting of conducting questionnaires and semi-structured interviews complemented by secondary sources and document analysis.
Operationalisation of the dependent variable through a set of democratic functions that the ECI might perform allows for a comprehensive assessment of its effectiveness, taking into account the various stages the mechanism is composed of.Such research design makes it also possible to identify the tool's limitations which might lie in its very design enshrined in Regulation 211/2011 10 , but might also refer to the insufficient handling of the ECI by the EU institutions, or be located at other -national or local levels.Finally, it provides a robust ground for presenting recommendations for ECI improvement, both from the perspective of its organisers as well as the EU institutions, with a special focus on its Europeanising function.
While this paper focused on the question of effectiveness of the ECI, there are other interesting aspects of the mechanism that open avenues for further research.One of them is the impact of domestic governance arrangements on national ECI-related mobilisation.In this sense, the research would focus on intra-state factors (institutional, political, social or economic) which might limit or facilitate the development of a successful ECI.Other interesting questions to address are what influence do organised interest groups have on forming the ECIs, and, on the other hand, what is the ECI's impact on the Brussels landscape of interest representation.An attempt to address these issues will surely yield interesting results.

Finally
, with respect to the agenda-setting function, one would firstly have to track the Commission's responses to the subsequent ECI's and identify what concrete measures have been undertaken to address citizen's needs.Identifying the issue salience and policy--makers' interest in the Commission's DGs or the EP would be another step.This would mainly require interviewing the EU officials from various institutions (possibly involving the European Economic and Social Committee and the Committee of the Regions).