Evaluation of innovative ideas for Public Transport proposed by citizens using Multi-Criteria Decision Analysis (MCDA)

Introduction: The use of participatory techniques in the field of transport is coming to the forefront recently. In this frame, eight co-creation workshops and five online crowdsourcing campaigns took place in Thessaloniki, Southern Tuscany, Rotterdam/The Hague, and Frankfurt, from which many innovative ideas to enhance Public Transport were generated by citizens. Purpose: A simple list of innovations would not be very useful for Public Transport Operators, as they cannot implement all of them at once. There was an obvious need for their ranking and this is the purpose of this paper. Methods: The ranking was realized with the most used Multi-Criteria Decision Analysis method in transportation research, i.e. the Analytic Hierarchy Process, using three criteria: Feasibility, Utility, and Innovativeness. An online questionnaire was distributed to experts, using a modified snowball sampling technique, which yielded 97 completed questionnaires. Results: Utility (42.90%) was found to be the most important criterion, followed by Feasibility (40.10%), and Innovativeness (17.00%). Four lists of innovations were derived, ranked with respect to a) all three examined criteria, b) Feasibility, c) Utility, and d) Innovativeness. The highest ranked innovation for a) and c) was found to be Mobility as a Service and platform with real-time travel, comfort, and multi-modal information; for b) City marketing from a Public Transport perspective; and for d) Advanced e-ticketing system. Conclusion: The results revealed which of the innovations are the most promising and provide valuable insight into how to integrate innovation with Public Transport to make it more attractive. Public Transport Operators may use the results according to the peculiarities of their city and the importance they give to Feasibility, Utility, and Innovativeness.


O R I G I N A L P A P E R Open Access
Evaluation of innovative ideas for Public Transport proposed by citizens using Multi-Criteria Decision Analysis (MCDA)

Introduction
The growing urban population worldwide and the traffic congestion resulted from the increase of car ownership during the last decades are urging for a shift to more sustainable and collective transport modes. In order Public Transport (PT) to increase its modal share, it has to attract new passengers and satisfy adequately the changing needs of the current ones, by introducing innovations and new services. Huge investments are required for this purpose and Public Transport Operators (PTO) are consequently dealing with the need of hierarchizing their priorities, focusing more intensely on the needs of their customers.
The use of participatory techniques in the field of transport is coming to the forefront recently, following a general trend to shift towards a more client-centered approach and put emphasis on services, business models, and innovation. Among the most widespread approaches to include "every idea coming from other fields, and it is currently not applied in Public Transport or even any idea that, while it has been conceived and may already been implemented in some areas of the world, it has not been yet spread or adopted in a certain local or/and national context" [1].
In an attempt to find compromising solutions to meet the different, and often conflicting, objectives of the multiple actors involved in any decision-making process, flexible methods and tools are needed. Under the objective of evaluating the output of the aforementioned collective intelligence processes that were applied in the context of CIPTEC, Multi-Criteria Decision Analysis (MCDA) methods were employed. MCDA methods have gained wide acceptance due to the fact that they embody many quantitative and qualitative variables and the characteristics of several scenarios can be assessed simultaneously [2]. The use of MCDA in the field of transportation has shown a steadily increasing trend during the last years. There is no single way for conducting a MCDA; there is a large number of methods and techniques with the aid of which it could take place. The suitability of the method strongly depends on the research objectives and the scope of the analysis [3].
The MCDA method used in the frame of CIPTEC project is the Analytic Hierarchy Process (AHP), one of the most commonly MCDA methods used in the field transportation research. The main reason for that is that through pairwise comparisons of the selection criteria and the alternatives, AHP leads to an, as far as possible, unbiased hierarchy of the alternatives (i.e. of the suggested innovations in the frame of CIPTEC). A hierarchy of the alternatives is always useful for the decision-maker especially in cases such as Public Transport where PTOs cannot implement all the suggested innovations at once but, instead, they should focus on the most important (i.e. on those with the highest impact).
The remainder of the paper is structured as follows: the literature review is presented in Section 2, followed by the description of the methodological framework in Section 3 and the survey design in Section 4. After that, the main findings of the analysis of the survey are displayed and discussed in Section 5. The paper concludes in Section 6, with a critical overall analysis of the results and directions for future research.

Literature review
Two main participatory techniques were used to collect innovative ideas in the field of Public Transport: co-creation workshops and crowdsourcing campaigns.
Co-creation may "refer to any act of collective creativity, i.e. creativity that is shared by two or more people" [4]. More specifically, co-creation is a revolutionary user-centered, collaborative approach, where a multitude of stakeholders (e.g. users, professionals, firms, etc.) are involved in the process of designing a product or service aiming to jointly create value [5]. Co-creation is an active, continually changing process, as it involves interactions between the relevant stakeholders [6]. The most important aspect of co-creation methods is the fact that they are user-centered. When the users are involved in the design of a product or service, the end value is usually increased due to the adaptation of the product/service to the users' needs. Moreover, among the most important benefits of the application of co-creation methods are the following: better and more effective decision making, reduced costs by being in line with the users' requirements, increased product quality, competitive advantage, customized products/services, and better customer needs satisfaction [7]. Considering the fact that the most users of Public Transport are loyal and captive, the inclusion of non-users in the co-creation process is desired, whenever insights on how they would shift to Public Transport are required. Despite the fact that co-creation processes were implemented in the field of urban planning during the past decades, their expansion in the Public Transport sector was not as widespread as it could be. Actually, it seems that the most cited relevant paper is not even about urban Public Transport but about the Swiss Federal Railway [8]. However, although PTOs have been using the traditional marketing approaches since years ago, new user-centered approaches should be investigated. As a consequence, new marketing approaches, like co-creation workshops can contribute to the increase of Public Transport attractiveness enabling the identification of innovative concepts/ideas that could be further developed by the experts [9]. Perhaps the best way to apply co-creation methods are co-creation workshops in which users, and sometimes non-users like in the case of Public Transport, participate. Eight (8) such co-creation workshops took place in the frame of CIP-TEC project, two (2) in each one of the following cities: Thessaloniki, Greece; Southern Tuscany, Italy; Rotterdam/The Hague, the Netherlands; and Frankfurt, Germany. The co-creation methods applied in these workshops and their results have been extensively presented in a relevant paper [10]. From these workshops came some of the innovative ideas that are evaluated in the frame of this paper.
Crowdsourcing campaigns were also used by CIPTEC to gather innovative ideas for Public Transport [11]. The term "crowdsourcing" was coined by Howe in 2006 who claimed that companies were using Internet outsourcing, thus the work to the crowd [12]. Crowdsourcing can be better understood as a field that combines three key elements: the "crowd", outsourcing, and advanced Internet technologies [13]. In 2008, Howe suggested the following definition: "Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call" [14]. Although the concept of crowdsourcing for innovation is relatively simple, it has been enabled during the last few years through the vast penetration of Internet technologies and smartphones in the market, but still its relevant literature is somehow limited. Most of the crowdsourcing for innovation examples are related to private sector product design and innovation process (e.g. Ford New Innovative Ideas [15] and IdeaConnection [16]). In many industry sectors (e.g. manufacturing, transport/warehousing, accommodation and food services, public administration, etc.) crowdsourcing was applied by big brands [17], however the examples of using crowdsourcing in Public Transport are not as many as in other domains. An indicative example is the "Bombardier Transportation", the rail equipment division of the Canadian firm Bombardier Inc., which has held three crowdsourcing initiatives: Innovation Express (internally for employees), YouRail (external contest), and YouCity (innovation contest related to the future of urban mobility) [17]. Another interesting crowdsourcing contest initiative, for developing innovative ideas in order to improve Public Transport in the South Holland region, is the Slim OV-idee [18]. Nevertheless, crowdsourcing for innovative ideas regarding Public Transport sometimes is incorporated in smart cities crowdsourcing events (e.g. hackathons) or platforms. Through crowdsourcing, innovative ideas can be gathered faster and cheaper from a bottom-up and more democratic process than internally within an organization that creates certain thinking patterns. After comparing different ideas generated both by the crowd and professionals, the ideas of the professionals proved to be less innovative and had less benefit for the end-user; on the other hand, they were more feasible (i.e. implementable) [19,20]. In addition, ideas generated by the crowd can be more commercially successful compared to ideas generated internally in an organization [21]. Although Public Transport affects a large and loyal community of people in any city and its services gain the interest of public, crowdsourcing for innovative ideas has not been widely used so far [11], apart from some exceptions [22,23]. Most research on Public Transport and crowdsourcing is about information provision crowdsourcing for use in real-time mobile applications and not about gathering innovative ideas to be integrated into the Public Transport system. Five (5) different online crowdsourcing campaigns were designed and ran in the frame of CIPTEC project, four (4) in the aforementioned cities in the respective national languages while the fifth one was addressed to all European citizens regardless the city they live in and it was launched in English [11]. The co-creation methods applied in these crowdsourcing campaigns and their results have been extensively presented in a relevant paper [11].
Both the co-creation workshops [10] and the crowdsourcing campaigns [11] were fruitful and many innovative ideas for Public Transport were gathered using these collective intelligence participatory techniques. Obviously, there was a need to evaluate and hierarchize these innovative ideas and this is the purpose of this paper. Since the innovative ideas we gathered are abstract ideas and not projects to be realized, conventional evaluation tools such as Cost Benefit Analysis (CBA) or Cost-Effectiveness Analysis (CEA) were not applicable, as they "are constrained by the difficulty in quantifying non-market impacts and monetising total costs and benefits" [24]. Moreover, MCDA methods are increasingly used due to the complexity of issues and the inadequacy of conventional tools, such as CBA and CEA, and it "allows for participatory analysis and qualitative assessment" [24]. Macharis and Bernardini (2015), in one of the most comprehensive reviews of the use of MCDA methods for the evaluation of transport projects, retrieved 276 publication titles in the period 1985-2012 that cover a broad scope of MCDA applications on transport projects [25]. Their analysis clearly shows that the use of MCDA methods on transport projects increases, especially after 1990. During that period, the transport projects examined were "for 22% passenger transport oriented, for 15% freight transport focused, for 11% transport technology and for 52% of 'general' type" [25]. A closer to per transport mode classification showed that "mobility management has the highest application (22%) after the general group (25%) followed by infrastructure (11%), Public Transport (10%) and technology (10%). Minor applications can be seen for the sub-categories: Bike (1%), Rail (3%), Air (5%) and as well 5% for Water (Inland Waterways and Maritime transport)" [25]. Since our focus is on Public Transport, there were publications focusing on: automatic metro map layout using multi-criteria optimization [26], on AHP-based decision making process for construction of public transportation city model: case study of Jeju, Korea [27], on spatial multiple criteria decision analysis in integrated planning for Public Transport and land use development in Klang Valley, Malaysia [28], on a multi-criteria evaluation model of Public Transport networks [29], and on a multistakeholder multi-criteria assessment framework of mobile payments illustrating the Swiss Public Transportation Industry [30]. Since 2012, and even 2015 when the review [25] was published, more papers on MCDA methods applied on transport projects have been published, e.g. [31,32], but as far as we know there was no paper applying a MCDA method on innovative ideas for Public Transport.
The final question is about which MCDA method should we use? There are many MCDA methods applied in transportation projects. Broadly, the MCDA methods are classified as follows: a) priority or utility methods, such as the Multi-Attribute Utility Theory / Multi-Attribute Value Theory (MAUT/MAVT) and the Simple Multi-Attribute Rating Technique (SMART); b) outranking methods such as the Elimination and Choice Expressing Reality, or "ELimination Et Choix Traduisant la REalité" (ELECTRE) in French, and the Preference Ranking Organization Method for Enrichment of Evaluations (PROMETHEE); c) distance based methods such as the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and the Multi-criteria Optimization and Compromise Solution, or Višekriterijumsko Kompromisno Rangiranje (VIKOR) in Serbian; d) mixed and other methods such as the linear additive method, the Analytic Hierarchy Process (AHP), the Likert scale, the fuzzy-TOPSIS, the fuzzy-VIKOR, etc. [31]. Actually, there are more than a hundred MCDA methods [25]! Different researchers have used different multi-criteria analysis methods for combining the value of performance parameters [31]. Mardani et al. (2015) showed that among the different MCDA methods (100%), AHP (32.57%), hybrid Multiple-Criteria Decision-Making (MCDM) (16.28%), aggregation Decision-Making (DM) methods (11.70%), TOPSIS (11.4%), ELECTRE (8.65%), PROMETHEE (6.62%), and VIKOR (3.56%) were applied in different kinds of MCDA problems between 2000 and 2014 [33]. Macharis and Bernardini (2015) found that the MCDA method mostly used in transport projects is the AHP method, as from the total amount of 276 publications they examined during the period 1985-2012, "33% revolves around the application of AHP or develops a variant to this approach applicable in the transport project field" [25]. Mardani, Zavadskas, Khalifah, Jusoh, and Nor (2016) reviewed a total of 89 papers, published from 1993 to 2015, from 39 high-ranking journals and found similar results: "AHP and Fuzzy-AHP techniques (25.84%) and its applications have been used more than other individual approaches" in transportation systems [34]. Obviously, AHP is the most commonly used MCDA method in the field of transport.
AHP is a non-linear framework for addressing complex semi-structured decision-making problems, introduced by Thomas Saaty in the 1970s [35]. It is a scaling method for deriving priorities (weights) for a set of options/scenarios based on their importance. The method uses a hierarchical or network structure to represent the problem in question and then the relations within this structure are built using pairwise comparisons [36]. AHP relies on three main principles: a) the decomposition of the decision space into its fundamental elements, b) the comparative judgments, and c) the synthesis of priorities [37].
The hierarchies of objectives and sub-objectives that are created attempt to reflect and incorporate the knowledge, experience, and aspirations of the decision-makers regarding the problem in discussion [38]. Comparative judgments principle refers to the development of a sound and comprehensive base for deriving priorities among the parameters involved in the process. Local priorities are obtained by comparing each node against each one of its peers with respect to its parent node.
In order to assist the decision-makers in carrying out the pairwise comparisons, Saaty created a nine-point intensity scale of importance, known as the fundamental scale of preferences, ranging from equal importance (1) to extreme importance (9). Technically, this is achieved by forming pairwise comparison matrices, where the ratio a ij , assigned by the decision-maker, expresses the dominance relation of the element in row i over the element in column j. These preference scores are then subject to a synthesis process; relative priorities (weights) are attained as the right principal eigenvector λ max of the pairwise comparison matrix. In cases when the transitive property holds, the aforementioned matrix is consistent; however, in real-life problems this is a very rare and unlikely thing to happen. Therefore, AHP suggests a specialized index, the Consistency Ratio (CR), to investigate the existence and degree of inconsistency of the judgments. In case CR is equal or lower of 10% the inconsistency is considered "acceptable", in the sense that its influence on the results is negligible or at least not of significant importance [36,39].
A mathematic approach of how the AHP method works follows, based on a comprehensive AHP note [40], which is based on Saaty's introductory book to the AHP method [35]. More specifically, the AHP method is implemented in three steps: 1) Computing the vector of criteria weights, 2) Computing the matrix of alternatives' scores, and 3) Ranking the alternatives. Let m be the number of the evaluation criteria and n the number of the alternatives to be evaluated [40].
In the first step, in order to compute the weights for the criteria, the AHP starts creating a pairwise comparison matrix A, which is an m × m real matrix, where m is the number of the evaluation criteria. Each entry a jk of the matrix A represents the importance of the jth criterion relative to the kth criterion. Obviously, if a jk > 1 the jth criterion is more important than the kth criterion, while if a jk < 1 the jth criterion is less important than the kth criterion. In case two criteria have the same importance, then the entry a jk = 1. The entries a jk and a kj satisfy the following constraint [40]: and, obviously a jj = 1 for all j. The relative importance between two criteria is measured by the aforementioned fundamental scale of preferences, as shown in Table 1, where it is assumed that the jth criterion is equally or more important than the kth criterion. The values in the matrix A are by construction pairwise consistent, see (1), but the ratings may show slight inconsistencies which do not cause serious difficulties [40]. From the matrix A, once it is built, it is possible to derive the normalized pairwise comparison matrix A norm by making equal to 1 the sum of the entries on each column. Each entry a jk of the matrix A norm is calculated as [40]: Finally, the criteria weight vector w (which is an m-dimensional column vector) is built by averaging the entries on each row of the matrix A norm , i.e. [40]: In the second step, the computing of the alternatives' scores, which is an n × m real matrix S, takes place. Each entry s ij of the matrix S represents the score of the ith alternative with respect to the jth criterion. In order to derive the scores, a pairwise comparison matrix B

MAAS
Mobility as a Service and platform with real-time travel, comfort, and multi-modal information Enabling a seamless journey, using the best possible connections and the most suitable transport modes, provided through one interface, such as a platform that brings together different types of transport options and the provision of real time transport information.

FLEXPRIC
Flexible pricing options Provision of customized pricing options for people with different transportation needs (e.g. individual travelers, business travelers, tourists, international guests, etc.).

GRTECH
Green platforms and "green technologies" systems at Public Transport stops providing comfort and pleasure to the passengers The idea is to create green spaces on the platforms and to install green technology systems, such as solar panels, at the PT stops for electricity production, creating thus a pleasant atmosphere.
PTSEATSAPP "Public Transport seats" app An app for the passengers to find out which parts of the vehicle are less crowded and more likely to have available seats in order to prefer the respective entrances, using sensors mounted in the seats.
PTWIDGET Public Transport widget indicating the accessibility of places of interest by Public Transport This is an indicator especially developed for companies and institutions to add to their website that indicates how well their location is connected to PT, promoting thus PT use.
ADVETICKET Advanced e-ticketing system Travel with PT without ever checking in or out, using mobile phones' Bluetooth beacons that will detect when you enter or exit the vehicle, calculate the travel cost, and display it on a web application.

MOBCHARLOCK Mobile phone charging locker
The charge-before-departure lockers are designed to safely fast charge mobile phones while waiting for the next train.
ONDEMAND "On demand" transport services using minibuses (demand responsive) Through these services, travelers can submit a travel request (origin, destination, and time frame) and all submitted requests are grouped spatially and temporally and minibuses are routed to serve them.

BLIP Bus lane with intermittent priority
This is a dynamic bus lane which operates as a bus lane only when a bus is using it; during its operation both vertical & horizontal marking will be activated to inform drivers to use a different traffic lane.

RETIJOURNAPP Real-time journey planner on app including management of customer requests
The app provides real-time info-services, including a dynamic journey planner, taking into account the current localization and service conditions, using social media as the main communication channel between PTOs and the users/citizens.

PTEVENTICK
Integrating Public Transport use in entrance tickets of events Entrance tickets for social events (e.g. concerts, sports, museums, etc.) will include the use of PT for the whole day free of charge.  Table 1 may be used to translate the pairwise evaluations into numbers. Following, the AHP applies to each matrix B (j) the same two-step procedure described for the pairwise comparison matrix A, i.e. it divides each entry by the sum of the entries in the same column, and then it averages the entries on each row, obtaining thus the score vectors s (j) , j = 1, … , m. The vector s (j) contains the scores of the evaluated alternatives with respect to the jth criterion. Finally, the score matrix S is obtained as: i.e. the jth column of S corresponds to s (j) [40].
In the third and final step, once the weight vector w and the score matrix S have been computed, the AHP obtains a vector v of global scores by multiplying S and w [40]: The ith entry v i of v represents the global score assigned by the AHP to the ith alternative. As the final step, the alternatives ranking is accomplished by ordering the global scores in decreasing order [40].
Regarding the aforementioned consistency, the AHP method incorporates an effective technique for checking the consistency of the evaluations made by the decision maker when building each of the pairwise comparison matrices involved in the process, i.e. the matrix A and the matrices B (j) . This technique relies on the computation of a suitable Consistency Index (CI), and will be described only for the matrix A. It is easy to adapt it to the case of the matrices B (j) by replacing A with B (j) , w with s (j) , and m with n. The CI is obtained by computing the scalar x as the average of the elements of the vector whose jth element is the ratio of the jth element of the vector A · w to the corresponding element of the vector w and then the CI is calculated by the following equation [40]: A perfectly consistent decision maker should always obtain CI = 0, but small values of inconsistency may be tolerated. More specifically, if: the inconsistencies are tolerable, and a reliable result may be expected from the AHP. In (8), RI is the Random Index (RI), i.e. the CI when the entries of A are completely random [40]. The values of RI are related to the number of m and may be found in the relevant literature. AHP is a well-established procedure that can be repeated; it allows performing consistency checks, both quantitative and qualitative data can be used as input and it is appropriate for group decision making. Moreover, as mentioned previously, it is applied widely in the academic literature. An additional advantage of AHP is that it tends to highlight slight differences among the examined alternatives which decision-makers are not always able to comfortably perceive [41,42]. For all these reasons, AHP is considered a smart choice when multidimensional decision problems are examined.

Survey design
In order to evaluate the collective intelligence processes results, derived, as discussed above, from the CIPTEC co-creation workshops and the crowdsourcing campaigns, an online questionnaire survey was designed and conducted. Twenty (20) innovations, comprising a combination of the innovations which were shortlisted as the most promising in each of the aforementioned cases, were included in this survey to be further assessed against a number of additional criteria. The 20 innovations (abbreviations, full names, and descriptions) are presented in Table 2.
The selection of a proper set of criteria is a very important step towards designing a comprehensive and trustworthy MCDA evaluation framework. The European Institute of Innovation & Technology (EIT) had launched a call for Knowledge and Innovation Communities 2018 and evaluation criteria such as: strategic approach, business model and financial plan, synergies, impact on the societal challenge, human capital, job creation, and economic growth were used [43]. Balm et al. [44] evaluated innovative urban and interurban freight transport solutions based on: costs and benefits to society, integration of stakeholders' opinions, and financial viability. According to the so-called "FAN" method, from the Synectics problem-solving tool, a recommended set of criteria to evaluate any innovative idea should include feasibility, attractiveness, and novelty [45]. Similarly, Terwilliger [46] suggested the Strategy, Need, Impact, Feasibility, Feel (SNIFF) test for innovation decision-making. Schöllhammer [47] believes that new, creative ideas should be evaluated with respect to how feasible, effective, efficient, and profitable they are.
It was decided to keep the number of criteria low and to include criteria with a rather generic nature than more specific ones, in order to keep the questionnaire as simple as possible, ensuring, thus, that it would appear more "attractive" to the survey participants. This way, we avoided the creation of a complex survey which could involve the risk of discouraging people to complete it, considering it time-consuming. After considering possible combinations of evaluation criteria based on the relevant literature and the limitations of the AHP method, three (3) main criteria were selected as the most suitable for this case: utility, feasibility, and innovativeness. The AHP questionnaire was distributed online by the twelve (12) CIPTEC project partners from seven (7) different European countries to experts, through e-mails to their contact lists and through thematic social-media groups (i.e. using a modified snowball sampling technique), in order to collect their views about the innovative concepts that had emerged both from the cocreation workshops and the crowdsourcing campaigns. The online survey took place in June 2017 and yielded 97 fully completed questionnaires, while the respondents were experts of several European countries. The original questionnaire (Fig. 1) was prepared in the English language, however translation in four (4) other languages was provided (viz. Greek, Italian, Dutch, and German, corresponding to the national languages of the countries where the co-creation workshops took place), in an attempt to make people more eager to participate.

Results and discussion
After the collection of the 97 questionnaires, the next step was to aggregate the different answers by calculating the rounded up geometric means. The aggregated pairwise comparison matrix, which was used for the weight elicitation, is presented in Table 3. Table 3 reveals that, according to the survey participants, there are no intense differentiations among the three criteria, as the maximum value that appears in Table  3 is the number 3. It is worth mentioning that, although  the topic of the survey is innovation in Public Transport, no precedence was given to the criterion of Innovativeness.
The weights (%) that were derived from this elicitation are displayed in Fig. 2. The most important criterion appears to be Utility (42.90%). Feasibility follows with a very small difference scoring (40.10%), a value close to that of Utility, leaving Innovativeness (17.00%) notably behind, which reflects a rather "sensible" and "rational" approach to Innovation.
As mentioned earlier, although the topic of the survey is innovation in Public Transport, no precedence was given to the criterion of Innovativeness. It appears like the radicalness of an innovation is a little sidelined in favor of the two more realistic criteria. This could be justified due to the fact that these ideas and concepts have resulted through a participatory process with the aim of encouraging innovation and triggering creativity. In this sense, innovativeness already comprises an inherent attribute of the selected ideas and, thus, experts may have placed impulsively more emphasis on the practical aspects of the concepts, drawing upon their expertise and knowledge of the real world. In any case, the motivation behind this result is worth further investigation.
Moreover, CR was calculated, and it was found to be approximately 2%, way under the 10% threshold suggested by Saaty [36]. The aggregated answers were then used as input to a specialized decision support software that implements AHP (Super Decisions by Creative Decisions Foundation [CDF]) [48].
Initially, the evaluation took place taking into account all three criteria (Utility, Feasibility, and Innovativeness), leading to the total ranking of the 20 innovations. Table 4 includes the innovations ranked in descending order, with the one which has obtained the highest evaluation score being on the top. Each innovation corresponds to two numbers, forming two different columns. The column named "Normals" comprises the evaluation score of each innovation that sum up to 1.00 (i.e. the percentage score of each innovation in total) while the one named "Ideals" is the same score scaled to the innovation with the highest score considered to have been given an evaluation score of 1.00.  The Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] is the innovation with the highest overall score (100%), with respect to which all others are compared. The second one, which gets a remarkably high overall score, is the Fully Equipped Public Transport making travel time exploitable time [FULLEQUIPPT] (99%). There are three more innovations that are considered to have an indisputable advantage compared to the rest (having obtained an ideal score of over 90%) and these are: City marketing from a Public Transport perspective [CITMARK], Integrating Public Transport use in entrance tickets of events [PTE-VENTICK], and Real-time journey planner on app including management of customer requests [RETIJOURNAPP].
Another sub group of three innovations follow, the overall score of which range between 73%-75%: Innovative solutions for on-board and at bus stops passengers' counting [PASSCOUNT], Use of data mining tools for enhancing service operation and performance [DATA-MIN], and Advanced e-ticketing system [ADVETICKET].
At the same time, others such as the Public Transport funding by the beneficiaries of the system [PTFUND], the Use of E-ink technology for the screens of info-panels [EINK], and the Umbrella brand concept [UMBRRAND] innovations score particularly low (below 40%) not having succeeded in attracting effectively the participants' attention. The rest of the innovations correspond to an evaluation score placed from approximately 40% to 60%.
From these observations, it is hence derived that the respondents are more positive towards an innovation which corresponds to a generic concept which has already been launched in some cases and has gained increasingly wide popularity recently: the Mobility as a Service (MaaS) concept. In addition to that, they appear eager to adapt innovations that aim to make their travel experience more convenient, constructive, and/or  amusing. They seem to focus more on innovations that they believe would have a direct impact on their commute, rather than on those which would possibly enhance Public Transport system in the long-term but would require a potential money contribution by them in the short term (e.g. the Public Transport funding by the beneficiaries of the system [PTFUND] innovation ends up in the last place). Another interesting outcome from the overall ranking is that the City marketing from a Public Transport perspective [CITMARK] innovation gathered noteworthy attention while the rather similar Umbrella brand concept [UMBRRAND] innovation ended up third from the end; the marketing idea seems additionally attractive when it is accompanied by a more holistic, urban approach. Despite the fact that the overall ranking allows a clear comparison among the preferences of the survey participants, in order to gain insight in the strong and weak aspects of the innovations with respect to each one of the examined criteria, the AHP was implemented three more times to rank the innovations based on Feasibility, Utility, and Innovation, respectively. This way, a sensitivity-like analysis was performed; the stability of the overall ranking was investigated through examining the impact of each criterion to the final outcome.
The ranking of innovations based exclusively on the criterion of Feasibility is illustrated in Table 5 and Fig. 4.
An interesting outcome is that Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] is no longer in the first place; in this case, the innovation which outperforms the rest (and is used to scale the others, considered as having "1.00" as a score) is the City marketing from a Public Transport perspective [CITMARK] innovation, which was previously ranked third in the overall ranking.
Fully equipped Public Transport making travel time exploitable time [FULLEQUIPPT] remains in the second place. Another difference between the two diagrams ( Figs. 3 and 4) is that the scores obtained in this latter case are considerably lower and, moreover, the scores of the last ones are also even lower than before. Public Transport funding by the beneficiaries of the system [PTFUND] does not manage to move from the last place in this ranking either; nevertheless, the 19th and 18th place are occupied  that significantly high scores over 90% are met in this ranking also, likewise with the first (overall) one. The ranking of innovations with respect to the last criterion (Innovativeness) is displayed in Table 7 and Fig. 6.
The first innovation here is different from the other three cases; it is the Advanced e-ticketing system [ADVE-TICKET], leaving the Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] concept in the second place. The Funding the Public Transport system through its beneficiaries [PTFUND] innovation is once again in the final place. It is worth mentioning that the City marketing from a Public Transport perspective [CITMARK] innovation, which was placed higher in the other cases is somewhere in the middle this time (9th). The Innovative solutions for on-board and at bus stops passengers counting [PASSCOUNT] innovation climbs up to the top three.
The evaluation with respect to each specific criterion has proved that the AHP analysis results can be considered stable and consistent as no drastic differentiations among the innovations ranking were identified. Figure 7 presents an accumulative diagram with the evaluation scores (i.e. Normals in %) of all the criteria per innovation that sum up to 100%. The trend-line has no actual meaning (i.e. it represents no physical trend but just the totals from the 1st to the 20th innovation). Figure 8 presents the previous information in the form of a radar chart. The radar/spider chart is a graphical method of illustrating data in two dimensions using axes with a common starting point. All axes are radially allocated with the same distance among them. The use of different colors is helpful in order to easily compare the alternatives with respect to the different criteria. In this case, it could be said that Utility and Feasibility have a greater buffer zone than Innovativeness; hence, Innovativeness is an important criterion for a lower number of innovations compared to the other two criteria.
Finally, for easy referencing, Table 8 presents the previous information in numbers.

Conclusions and perspectives
The analysis and synthesis of the obtained results revealed very interesting observations regarding which of the innovations can be considered as the most promising among the ones that resulted from the collective intelligence processes applied in the context of CIPTEC project. Noteworthy similarities, but also contradictions, emerged among the overall ranking of innovations and the ranking with respect to each one of the examined criteria. The approach presented herein provides valuable insight into mobilizing innovation, in the form of ideas "provided by citizens, for citizens", to make Public Transport in cities more attractive and hence increase its use. In order to contribute in achieving the challenging task of planning successfully innovative urban Public Transport systems for the years to come, the output of the survey described in the present paper will feed the development of the CIPTEC Toolbox for Public Transport Innovation, a "living-tool" to assist stakeholders and decision-makers in integrating innovation to urban policies, to co-shape with European citizens a bright, novel future for Public Transport [49]. Finally, the results of the present paper can give a first clue to PTOs on which innovations should be applied to each special case, depending on the peculiarities of each European city and the importance they give to Feasibility, Utility, and Innovativeness. Other techniques applied in the frame of CIPTEC project, such as the Conjoint Analysis, give even further insights, such as possible shift in Public Transport modal share and hidden groups amongst the Public Transport users and citizens of European cities [50].