SELECTION OF THE MOST ADEQUATE TRIP-MODELLING TOOL FOR INTEGRATED TRANSPORT PLANNING SYSTEM

This paper deals with the problem of selection the most suitable trip-modelling tool (TMT), which is a part of the more complex integrated transport planning system (ITPS) at the regional scale. Since an application of TMT is not autonomous and several different users exist the selection problem is not a trivial. In this paper, an original five-phase selection procedure is presented. The first phase consists in specification of both, detailed expectations of all identified users and technical requirements of ITPS. Second phase deals with research on available TMT while a third one is concentrated on defining a comprehensive set of criteria. In this phase critical criteria as well as selection criteria are defined. First one is utilised to eliminate unacceptable TMTs in phase four and second one to evaluate and select most adequate TMT in phase five. In the paper an exemplary application of this procedure is presented. The authors have defined 2 critical criteria and a set of 19 selection criteria. The last one is divided into 3 main subsets, i.e. functional, technical and financial contexts of selection process. All the selection criteria are characterised by 43 sub-criteria and some of them are more detailed extended. Using this procedure 3 out of 6 alternative TMTs including Emme, Aimsun and Visum have been initially accepted and next evaluated. Finally, Visum has been selected and recommended for application into ITPS.


Introduction
The Public Transport Act in Poland states the local authority at the regional level is responsible for organisation of public transport (Sejm RP, 2011).Their responsibility covers several activities, including planning, organizing and managing of the public transport in the region (at the voivodeship territory in this case).Meeting this obligation, the most of authorities rely on very simplified assumptions and very often it is inadequate to the complexity of defined duties.In fact, according to many expert's opinions regional administration should use more sophisticated solutions.In fact, different tools and methods can support in design of the most rational configuration of public transport system. Lee (2004) has presented a comprehensive review on different techniques applied in this matter.The authors of this paper in their previous works have also proposed and applied different techniques, e.g. a combination of expert knowledge and simulation (e.g.Bieńczak et al., 2015a) or optimisation of transport network (e.g.Bieńczak et al., 2015b).
In this paper a complex integrated transport planning system (ITPS), which is a decision support system based on two mutually communicated components i.e. the knowledge base (KB) and tripmodelling tool (TMT) is considered (see Fig. 1).First component is used to store structured information, which is both, input for current calculations and a set of output of such results.While KB is strictly customised to users' demand, TMT is typical software, which have to meet target user's requirements.Trip model assists transport engineers, specialists and decision makers in forecasting traffic flows and utilisation of transport infrastructure in a certain transport system.It helps to generate a set of evaluation measures useful for assessment transport system behaviour and its operation.These parameters can also be applied to construct alternative transport solutions.Typically, any TMT consists of two key interrelated components, including demand (i.e.passengers' travel needs and their motivations) and supply (i.e.transport capabilities, network structure, fleet, etc.).
Piotr Sawicki, Marcin Kiciński, Szymon Fierek Selection of the most adequate trip-modelling tool for integrated transport planning system 56 The problem considered in this paper is referred to construct a universal procedure for TMT selection to meet a several target users' needs on the one hand, and to involve TMT as a part of global system -ITPS, on the other hand.
The paper is composed of five sections and references.The first section is an introduction to the paper.In the section two a brief research on software selection techniques has been proposed.The concept of TMT selection procedure as a part of ITPS is presented in section three.An exemplary application of the procedure is demonstrated in section four, and the final conclusions and remarks are drawn in section five.

State of the art software selection
Software selection has been widely discussed issue in the literature.Many authors, define the guidelines and risk analysis of selecting enterprise resource planning systems -ERP (Spencer and Johnston, 2014; Panorama Consulting Solutions, 2016; Ayağ and Özdemir, 2007).They claim that many entities, before purchasing specialized software, make a comparison of tools available on the market.The methods used to identify a set of possible candidate solutions are, for the most part, subjective.The individual, or group of them, performing the assess-ment have various experiences that determine the decision process, either consciously or subconsciously.Other authors emphasize that a successful evaluation of computer tools should be made on the basis of a formalized procedure (Sai, 2004).This means that a whole process of the selection and evaluation should be established and documented in a manner, which allow repeatability.Bandor (2006) claims that beside the typical criteria, which are determined by the functional requirements, it is very important to take into account intangible factors and risk as well.Intangible factors are not the traditional "quality" factors, but rather characteristics that can affect on the overall utilizing of software by the individuals, e.g., competence interface language, ease of adaptation to changing requirements, fitness of features (avoidance of paying for many features that cannot be used), or open source code (possibility to modify software).Typically, after selection of evaluation criteria, a formal assessment mechanism has to be defined to compare all alternative software.Some authors (e.g.Lai, Wong and Cheung, 2002;Lai et al., 1999) use AHP method or balanced scorecard technique (Marr and Neely, 2003).Several works exists on using artificial neural network based on analytic network process approach (Yazgan et al. 2009).All of these methods allow comparing various products by using the selection criteria and assigning a value to the considered criteria.The software with the best result is the preferred one.Each criterion might have its own weight in the computation procedure, and the final result is a weighed score in such a case.Although the software selection is well known problem in the literature, the problem of trip-modelling tool selection has not been discussed yet.There are a few papers describing the selection of trip-modelling software, but they are mostly concentrated on micro-simulation traffic tools (e.g.Dowling et al., 2002Dowling et al., , 2004;;Maciejewski, 2010).In these circumstances, the authors of this paper have decided to propose a formal procedure and describe its application in a real world case.

A concept of the TMT selection procedure 3.1. Key phases of the procedure
A selection of TMT for integrated transport planning system is composed of five interrelated phases (see Fig. 2) including:

Phase 1 -Specification of detailed expectations for TMT
In the phase 1 ( ! ) a comprehensive and detailed review on target user's expectations of the trip modelling and management tool has to be extracted.At this stage several aspects have to be considered, including: capabilities of all TMTs available on the marketplace, current and predicted obligations, duties and tasks of all target TMT's user, a long term strategy for transport system development, data retrieved from the current transport system (e.g.data granularity, its adequateness and format).As a result of this phase, a set of structured user's needs with reference to TMT is defined.It is a key input for three consecutive phases - ! and  ! .

Phase 2 -Desk research on available TMTs
During phase 2 ( ! ) a desk research on TMTs available at the marketplace is conducted.At this stage of the procedure both, the set of structured user's needs (see  ! ) and TMT's producer or distributor offers are explored.Finally, a key result of  ! is a general set of functional features of TMT's.This result is an input for two other phases of the procedure, i.e.  ! and  ! .

Phase 3 -Defining evaluation criteria
Phase 3 ( ! ) is started with defined TMT's users expectations (see the result of  ! ), and is devoted to define two separate sets of criteria.First one is composed of critical criteria having a crucial meaning for the users.A lack of compliance with any of critical criteria should result in rejection of TMT from further analysis.Second set, is composed of selection criteria, which is utilised for thorough analysis of alternative TMTs that are positively verified with critical criteria first.Selection criteria are utilised for differentiate alternative TMTs and perform a final selection.While constructing the set of selection criteria, several practical aspects should be reflected, including: functional -, all the TMT's capabilities that help the users to achieve requested decision concerning transport system, technical -, which is TMT's specification that helps to communicate this tools with other facilities and data, i.e. standard cohesion, data sharing, computational technology etc., financial -, available budget, i.e. cost of TMT this example is to introduce the core elements, simplificae business process model provides a simplified view of how actually conducted.In addition, there are many authors any reviewers.For convenience, just one author and one As will be discussed below, situations in which multiple lved in the same role cannot be covered conveniently.example represent roles and not concrete participants in a h role at run time has multiple concrete participants who in the business process instance.The BPMN standard l represents a participant in a process.It is also acts as a aphical container for partitioning a set of activities from in the context of B2B situations."ts when the PC Chair is asked to organize the scientific nce.This is reflected by the start event of the process at vent is something that 'happens' during the course of a ese events a↵ect the flow of the process and usually have an impact (result).Events are circles with open centres rkers to di↵erentiate di↵erent triggers or results."cted first is the publication of a call for papers with den the conference, such as name and location, and also g the topics addressed by the conference.The receipt of event that is relevant for the process.This concept is rocess when the published call for papers activity sends uthor receives.Receiving this message is represented by e author process.The cause of this event is receiving the Since the goal of this example is to introduce the core elements, simplifications are in place: the business process model provides a simplified view of how review processes are actually conducted.In addition, there are many authors and there are also many reviewers.For convenience, just one author and one reviewer are shown.As will be discussed below, situations in which multiple participants are involved in the same role cannot be covered conveniently.The pools in this example represent roles and not concrete participants in a business process.Each role at run time has multiple concrete participants who are actually involved in the business process instance.The BPMN standard indicates that "a pool represents a participant in a process.It is also acts as a "swimlane" and a graphical container for partitioning a set of activities from other pools, usually in the context of B2B situations." The process starts when the PC Chair is asked to organize the scientific program of a conference.This is reflected by the start event of the process at the PC Chair.An event is something that 'happens' during the course of a business process.These events a↵ect the flow of the process and usually have a cause (trigger) or an impact (result).Events are circles with open centres Since the goal of this example is to introduce the core elements, simplifications are in place: the business process model provides a simplified view of how review processes are actually conducted.In addition, there are many authors and there are also many reviewers.For convenience, just one author and one reviewer are shown.As will be discussed below, situations in which multiple participants are involved in the same role cannot be covered conveniently.The pools in this example represent roles and not concrete participants in a business process.Each role at run time has multiple concrete participants who are actually involved in the business process instance.The BPMN standard indicates that "a pool represents a participant in a process.It is also acts as a "swimlane" and a graphical container for partitioning a set of activities from other pools, usually in the context of B2B situations." The process starts when the PC Chair is asked to organize the scientific program of a conference.This is reflected by the start event of the process at the PC Chair.An event is something that 'happens' during the course of a business process.These events a↵ect the flow of the process and usually have a cause (trigger) or an impact (result).Events are circles with open centres to allow internal markers to di↵erentiate di↵erent triggers or results." The activity enacted first is the publication of a call for papers with detailed information on the conference, such as name and location, and also information regarding the topics addressed by the conference.The receipt of a message can be an event that is relevant for the process.This concept is used in the sample process when the published call for papers activity sends a message that the author receives.Receiving this message is represented by the start event of the author process.The cause of this event is receiving the Since the goal of this example is to introduce the core elements, simplifications are in place: the business process model provides a simplified view of how review processes are actually conducted.In addition, there are many authors and there are also many reviewers.For convenience, just one author and one reviewer are shown.As will be discussed below, situations in which multiple participants are involved in the same role cannot be covered conveniently.The pools in this example represent roles and not concrete participants in a business process.Each role at run time has multiple concrete participants who are actually involved in the business process instance.The BPMN standard indicates that "a pool represents a participant in a process.It is also acts as a "swimlane" and a graphical container for partitioning a set of activities from other pools, usually in the context of B2B situations." The process starts when the PC Chair is asked to organize the scientific program of a conference.This is reflected by the start event of the process at the PC Chair.An event is something that 'happens' during the course of a business process.These events a↵ect the flow of the process and usually have a cause (trigger) or an impact (result).Events are circles with open centres to allow internal markers to di↵erentiate di↵erent triggers or results." The activity enacted first is the publication of a call for papers with detailed information on the conference, such as name and location, and also information regarding the topics addressed by the conference.The receipt of a message can be an event that is relevant for the process.This concept is used in the sample process when the published call for papers activity sends a message that the author receives.Receiving this message is represented by the start event of the author process.The cause of this event is receiving the Since the goal of this example is to introduce the co tions are in place: the business process model provides a review processes are actually conducted.In addition, t and there are also many reviewers.For convenience, ju reviewer are shown.As will be discussed below, situa participants are involved in the same role cannot be c The pools in this example represent roles and not co business process.Each role at run time has multiple con are actually involved in the business process instance indicates that "a pool represents a participant in a pro "swimlane" and a graphical container for partitioning other pools, usually in the context of B2B situations." The process starts when the PC Chair is asked to program of a conference.This is reflected by the start the PC Chair.An event is something that 'happens' business process.These events a↵ect the flow of the pr a cause (trigger) or an impact (result).Events are cir Since the goal of this example is to introduce the core elements tions are in place: the business process model provides a simplified review processes are actually conducted.In addition, there are ma and there are also many reviewers.For convenience, just one auth reviewer are shown.As will be discussed below, situations in whi participants are involved in the same role cannot be covered conv The pools in this example represent roles and not concrete parti business process.Each role at run time has multiple concrete partic are actually involved in the business process instance.The BPM indicates that "a pool represents a participant in a process.It is al "swimlane" and a graphical container for partitioning a set of act other pools, usually in the context of B2B situations." The process starts when the PC Chair is asked to organize th program of a conference.This is reflected by the start event of the the PC Chair.An event is something that 'happens' during the business process.These events a↵ect the flow of the process and u a cause (trigger) or an impact (result).Events are circles with o acquisition and its maintenance.The set of selection criteria should also reflect both aspects, the presence of multiple users (possibly different purposes of TMT use), and the context of TMT as a main component of more complex system -ITPS.It should also be noticed, that some criteria from the considered set are maximised type and others are minimised.A criterion is called a maximised when with the increased value of the criterion a preference is increased too.On the other hand a criterion is minimised when the higher value means lower preference.

Phase 4 -Critical elimination of TMTs
This phase is devoted to very first evaluation of alternative TMTs and its result is elimination from the further analysis all alternatives having a conflict with at least single critical criterion (see  ! ).To recognise such a conflict a set of functional features of all considered TMTs is necessary (see  ! ).

Phase 5 -Conducting the assessment and final selection
The last phase of the procedure ( ! ) is concentrated on a detailed evaluation of alternative TMTs and making the final selection.This phase is based on several inputs generated before, including set of functional features (see  ! ) and a set of the selection criteria (see  ! ).In the phase  ! a final recommendation for TMT selection is produced as a final result of the whole procedure.
The key assumption of this phase is to achieve the comparable measures in the set of considered selection criteria.It is done using normalisation of each criterion into the interval (0, 1).Based on that a final assessment of -th alternative TMT is performed using measure  !, see eq. (1). where: -value of -th maximised criterion for -th alternative TMT and transformed to inter-val (0, 1),  !" -value of -th minimised criterion for -th alternative TMT and transformed to interval (0, 1); and A transformation of the real value of -th maximised type criterion for -th alternative TMT into interval (0, 1), denoted as  !" , is calculated using formula (3), as: where:  ! !value of -th criterion for -th alternative TMT which is expressed in its natural scale,  ! !the lowest value of -th criterion, expressed in its natural scale,  ! !the highest value of -th criterion, expressed in its natural scale.
A transformation of the real value of -th minimised type criterion for -th alternative TMT into interval (0, 1), denoted as  !" , is calculated using formula (4), as: The graphical interpretation of an original value transformation into the interval (0, 1) is presented on the Fig. 3. Since the aim of this procedure is to evaluate how each alternative TMT is good for the specific needs, most of the criteria  ! are usually constructed with hierarchy structure.It means a final value of such a criterion  !( ! ) is a result of aggregation certain subcriteria, and is calculated using the formula (5), as follows: where:  ! !"#an aggregated value of -th criterion with respect to its sub-criteria at -th ( = 1,2, … , ), or -th ( = 1, 2, … , ) level of hierarchy, if applied, for -th alternative TMT; value is expressed in its natural scale (it is assumed the same scale at different level of hierarchy for a single th criterion).

General assumptions
The exemplary application of the proposed procedure for a selection the most adequate TMT for integrated transport planning system is performed for the Regional Rail Carrier (Łódzka Kolej Aglomeracyjna Sp. z o.o.-ŁKA) and The Office of the Marshal of the Łódz Province (Urząd Marszałkowski Województwa Łódzkiego -UMWŁ), a statutory owner of the ŁKA.For their individual expectations an application of the selection procedure is detailed characterised and presented in the following sections of this chapter.

Phase 1 -Specification of detailed expectations
Based on analysis of a documentation of statutory task of both target ITPS's users, i.e.ŁKA and UMWŁ, and mutually discussing their needs, a group of 8 categories of expectations have been identified.The following set of functionalities is expected from target TMT (Fierek et al., 2015): -A view of complex transportation system landscape, including: transportation infrastructure and coincidence of different modes, reporting on technical status of a infrastructure and passenger applied flow, location of interchange nodes and its current status, identification of bottlenecks and other malfunctions on the network, modal split status, profitability of the system and its lines, time vs. cost competitiveness of specific lines, transport accessibility for specific trip generators.
-A global view on impact of external factors (e.g. economy, sociology, psychology etc.) on specific global characteristic of a transport system, such as: modal split, passenger behaviour, and others.
-A simulation of results produced by transport system, including: changes on the network (e.g.quality improvement of the infrastructure, temporary speed restrictions, etc.) and organisational changes (e.g.ticket price, city access fee, new location of trip generator etc.).
-A matching analysis of the feeder transport with reference to the main lines, including modelling of time components of the journey (transfer time and riding time), generation of timetables of the feeder transport with respect to the main line schedule, timetables optimisation etc.
-Reporting mechanism on the cause-and-effect analysis in transport system for a specific time horizon.
-An analysis on dependencies between development of transport system and land use, including modelling key functions of the region (i.e.investment, building and commerce), and prediction of development intensity.
-Passenger traffic forecast calculations mechanism, including prediction of traffic with reference to the social and economic factors and result of wide spectrum of other research, on-line update of key input for modelling procedure (e.g.tariffs, fuel price, current status of the transport system).
-Ability of the TMT for a further extensions and setting up other functionalities of ITPS.Transformation scale of j-th minimised criterion

Phase 2 -Desk research on available TMTs
In the phase 2 ( ! ) of the procedure a market recognised TMTs have been taken into consideration.There are at least few major software vendors and or distributors offering some tools to build and manage traffic model now.Using available information one can say that none of the tool is definitely best or worst.Although, the tools of this type (analysed software) are manufactured by different entities and they have a number of common features and components used both in the construction and management of a trip model.As a result of market research six alternative tools have been selected (see Table 1).A common set of the characteristics of all those TMTs is composed of the following issues: mapping of the transport network (including several points, lines and zones, road and rail transport, local, agglomeration and regional transport), modelling of the demand and supply, assignment procedures, modelling of friction factor, construction and management of different scenarios, static and dynamic traffic distribution, modelling of the network parameters, use of geographic information system -GIS, interface for script-based applications, creating analyses, statistics and reports typical for management of the transportation system.sales representatives and after-sales support should be located in Poland, a producer or distributor of TMT have to declare an interests of sell their product to the considered users, i.e. for both, rail carrier -ŁKA and marshal office -UMWŁ.In the next step, the set of selection criteria for detailed evaluation of considered TMTs has been constructed.In this step 19 evaluation criteria have been defined, within 3 key evaluation contexts, including: The list of all selection criteria is presented in Table 2. Some of the criteria have their sub-criteria (43 items) and some of the sub-criteria are described more detailed, i.e. they have another level of decomposition (23 items).

Phase 4 -Critical elimination of TMTs
Using the set of critical criteria defined at phase  !, for further evaluation a few alternative TMTs have been extracted.Only those tools having sales representatives and after-sales support located in Poland on the one hand, and expressed interests of sell their product (TMT) for ŁKA and UMWŁ, on the other, have been considered for further evaluation.Thus, 3 out of 6 TMTs have been extracted, i.e.: -Aimsun, -Emme, -Visum.

Phase 5 -Conducting the assessment and final selection
Using the set of selection criteria (see result of phase  ! ), all 3 alternative TMTs have been evaluated accordingly.To achieve an objective opinion on functionality of each TMT a sequence of experimental modelling activities on reference transport system -RTS have been performed (see Fig. 4).Based on ex post assessment the value of each criterion has been determined and the results of all experiments, expressed in its natural scale, are presented in Table 3.

Subset of functional aspects of TMT -𝑭
1 Availability of transport system modelling, with respect to (0, 5)  1,1 private transport mode, including (0, 3) Availability of defining attributes of transportation network components, i.e. (0, 9)  2,1 for ways, including (0, 5) temporary shutdowns (e.g.road repairs, events etc.), for facilities, including (0, 4) Availability of parameters defining mechanism for transport lines, including (0, 6)  3,1 timetables for each line, 0 ∪ 1  3,2 time parameters, including (0, 3) modelling of different aspect of regional development, including (0, 4) Selection of the most adequate trip-modelling tool for integrated transport planning system 64 Fig. 4. A concept of TMT's assessment using reference transport system A comparison among all three alternative TMT and its comparison to ideal profile of TMT is presented on Fig. 5.Taking this result into account, one can say the most recommended TMT is Visum ( ! ), which is around 16% better than two other TMTs, i.e.Emme and Aimsun (17,9 vs. 15,0 points) and is only 6% worst than ideal profile of TMT (i.e.17,9 vs. 20 points).
Business Process Modeling Notation: categories of elements l of this example is to introduce the core elements, simplifica-: the business process model provides a simplified view of how are actually conducted.In addition, there are many authors so many reviewers.For convenience, just one author and one wn.As will be discussed below, situations in which multiple involved in the same role cannot be covered conveniently.this example represent roles and not concrete participants in a .Each role at run time has multiple concrete participants who olved in the business process instance.The BPMN standard pool represents a participant in a process.It is also acts as a a graphical container for partitioning a set of activities from ally in the context of B2B situations."starts when the PC Chair is asked to organize the scientific nference.This is reflected by the start event of the process at n event is something that 'happens' during the course of a .These events a↵ect the flow of the process and usually have ) or an impact (result).Events are circles with open centres l markers to di↵erentiate di↵erent triggers or results."enacted first is the publication of a call for papers with deon on the conference, such as name and location, and also rding the topics addressed by the conference.The receipt of e an event that is relevant for the process.This concept is ple process when the published call for papers activity sends the author receives.Receiving this message is represented by Business Process Modeling Notation: categories of elements l of this example is to introduce the core elements, simplifica-: the business process model provides a simplified view of how are actually conducted.In addition, there are many authors so many reviewers.For convenience, just one author and one wn.As will be discussed below, situations in which multiple involved in the same role cannot be covered conveniently.this example represent roles and not concrete participants in a .Each role at run time has multiple concrete participants who lved in the business process instance.The BPMN standard pool represents a participant in a process.It is also acts as a a graphical container for partitioning a set of activities from ally in the context of B2B situations."starts when the PC Chair is asked to organize the scientific ference.This is reflected by the start event of the process at n event is something that 'happens' during the course of a .These events a↵ect the flow of the process and usually have or an impact (result).Events are circles with open centres markers to di↵erentiate di↵erent triggers or results."enacted first is the publication of a call for papers with deon on the conference, such as name and location, and also rding the topics addressed by the conference.The receipt of e an event that is relevant for the process.This concept is ple process when the published call for papers activity sends he author receives.Receiving this message is represented by f the author process.The cause of this event is receiving the Since the goal of this example is to introduce the core elements, simplificaons are in place: the business process model provides a simplified view of how view processes are actually conducted.In addition, there are many authors nd there are also many reviewers.For convenience, just one author and one viewer are shown.As will be discussed below, situations in which multiple articipants are involved in the same role cannot be covered conveniently.The pools in this example represent roles and not concrete participants in a usiness process.Each role at run time has multiple concrete participants who re actually involved in the business process instance.The BPMN standard dicates that "a pool represents a participant in a process.It is also acts as a swimlane" and a graphical container for partitioning a set of activities from ther pools, usually in the context of B2B situations." The process starts when the PC Chair is asked to organize the scientific rogram of a conference.This is reflected by the start event of the process at e PC Chair.An event is something that 'happens' during the course of a usiness process.These events a↵ect the flow of the process and usually have cause (trigger) or an impact (result).Events are circles with open centres allow internal markers to di↵erentiate di↵erent triggers or results."The activity enacted first is the publication of a call for papers with deiled information on the conference, such as name and location, and also formation regarding the topics addressed by the conference.The receipt of message can be an event that is relevant for the process.This concept is sed in the sample process when the published call for papers activity sends message that the author receives.Receiving this message is represented by e start event of the author process.The cause of this event is receiving the

Fig. 1 .
Fig. 1.General idea of ITPS and the role of TMT as a key component of the system Source: based on Fierek et al., 2015
Notation: categories of elements

0 1
Original value of the j-th criterion Transformation scale of j-th maximised criterion 0 1

Fig. 5 .
Fig. 5. Partial evaluation and final selection based on value  !( -financial,  -technical and financial context) A better compared to both Emme and Aimsun results from having: mechanism supporting calibration process (c !" ), including standard calibration of the trip model and O-D matrix optimisation,-Polish version ( !" ), including user's interface, data and results sharing with other model standards ( !" ), including both macro-and microscopic models.What is more, Visum is better than Aimsun in having timetable generation mechanism ( ! ), including gene-

Table 1 .
Alternative TMT solution

Table 2 .
The set of selection criteria, sub-criteria and sub-sub-criteria

Table 2 .
The set of selection criteria, sub-criteria and sub-sub-criteria (cont.)

Table 4 .
Assuming an equivalence of each criterion, i.e. a value of each criterion is the same ( != 1, for  = 1, … ,19), the value of  ! and its partial values, i.e.  !!,  !! and  !! for each alternative TMT, have also been calculated.All these results are presented in Table4, as well.