A methodological framework based on a DANP model for evaluating the software quality in terms of usability: a preliminary investigation on mobile operating systems

Evaluation of the software usability as concerns software alternatives represents a typical Multi- Criteria Decision-Analysis (MCDA) problem. Such a kind of problem can be successfully faced via structured models and approaches which validity have been widely shown in literature. In this work, a methodological framework based on a MCDA model integrating the DEMATEL method along with the ANP technique is proposed and explained with more detail to address the relevant issue of the software usability evaluation. The applicability of the considered DEMATEL-ANP (DANP) based model in the under analysis contest is demonstrated via a preliminary investigation, where the usability of the two major Mobile Operating Systems (MOSs) is compared and quantitatively prioritized. Results of both performed comparison method and sensitivity analysis confirm the validity of the proposed MCDA model as well robustness of its outcomes. Moreover, results obtained point out about the influenced, independent and core factors for the MOSs usability, as well as strengths, weaknesses and discrepancies in the usability as concerns the two investigated alternatives. Representing these outcomes crucial aspects for driving focused MOSs usability improvement processes and strategies, they give also evidence of the effectiveness of the proposed methodological framework.


Introduction
Over time, the usability concept has received numerous definitions and, despite its great relevance in the software context, to date there is not a commonly accepted definition of its own nature and on features that a software must have to achieve its better outcomes. Evolution of the usability concept has led to establish that it is not an exclusive software feature, but it can be considered as a result of both the human-computer interaction and the context of reference (Shackel 1991). The ISO 9241-11:1994 provides the most considered usability definition in ergonomic standards. Usability is identified as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction, in a specified context of use. Effectiveness is intended as a measure of the achieving degree of fixed goals in a complete and accurate manner, efficiency a measure of difficulty with which the set objectives are achieved, and satisfaction a measure of comfort and ease-of-use. The more recent ISO/IEC 25010:2011 introduces a quality model suitable for both computer systems and software products including several relevant quality attributes. In such a quality model, the usability represents a crucial attribute which plays a dual role, the first one with regard to the quality of the final product, while the second one is related to the software design/development activities, being the usability feature their final goal. Methods and approaches developed in literature for evaluating the software usability are various and even relatively complex to adopt. According to Paz & Pow-Sang (2014) these methods can be mainly classified as survey, user testing, interview, prototype evaluation, software metrics/usability metrics, automated evaluation via software tool, focus group, and checklist verification. The choice of the method and/or related combinations which give the more expected relevant results, can be related to the fundamental factors characterizing the analysis background. They include resources required, evaluation type, phase in which the system is located and type of measurements and information provided. Given the lack of literature in this context characterized by poorly structured evaluation methods, whose validity and robustness are not proven in quantitative manner, the main purpose of the present paper is to propose a methodological framework for evaluating the software usability based on a Multi-Criteria Decision-Analysis (MCDA) model. In particular, this model integrates the DEcision MAking Trial and Evaluation Laboratory (DEMATEL) method (Gabus and Fontela 1972) along with the Analytic Network Process (ANP) technique (Saaty 1996). DEMATEL is able to analyze interdependencies between under analysis elements via proper cause-effect diagram. Conversely, ANP, which is the generalization of the well-known AHP method (Saaty 2008), is able to deal with real-world decision problems. Both herein considered methods just use human judgments as input data. In literature, DEMATEL and ANP are very often integrated to successfully solve complex and in subjective environment MCDA problems. In order to show how the proposed methodological framework works, its applicability and effectiveness in the under analysis contest, a preliminary usability study involving the two major Mobile Operating Systems (MOSs) was herein carried out. More in detail, according to viewpoints and opinions of a panel of experts specifically selected for dealing with this analysis, the key usability factors of MOSs were firstly found out and arranged within 5 usability clusters. Then, a MCDA investigation aimed at evaluating the usability as concerns the two considered MOSs was carried out. A comparative study based on the typical ANP and a sensitivity analysis were also conducted so as to prove the validity and robustness of results obtained via the proposed methodological framework.
The contributions to literature of this work concerns several fundamental aspects. In the first instance, as concern the performed preliminary usability investigation, by combining a comprehensive literature review and information provided by highly experienced and recognized experts in the field, the key factors and related clusters affecting the MOSs usability were synthetized and used to develop a conceptual model of the MOSs usability. Moreover, from a methodological viewpoint, a MCDA model combining both DEMATEL, to identify the cause and effect interrelationships of usability factors, and ANP to evaluate importance weights of decision elements, was proposed, and its applicability, validity, robustness and effectiveness in the under analysis contest were demonstrated. More in detail, via the performed empirical analysis, inner and outer dependency relationships between key usability factors, as well as strengths, weaknesses and discrepancies in the usability as concerns the two investigated MOSs were obtained and analyzed in a quantitative manner. These outcomes can support MOSs decision-makers and developers in pursuing improvement choices. In particular, such a knowledge can help them to understand the relevant factors of the MOSs usability and support focused MOSs usability improvement strategies and processes.
The remainder of this paper is organized as follows: the considered DANP-based model is described in section 2; section 3 illustrates materials and methods of the proposed methodological framework, while the performed usability analysis involving the two major MOSs is detailed in section 4. Finally, the conclusions section comprising a summary and the possible future research developments closes this work.

DANP-based model
DANP models take remarkable attention of decision analysis community in recent years and appear as one of the most promising approaches to handle criteria interactions in a MCDA setting (Gölcük & Baykasolu 2016).
As well known, ANP requires the description of a decision network comprising elements and clusters of elements and links between elements and clusters. Unlike a hierarchical structure unidirectionally developed from the top to the bottom, a decision network is distributed in all directions and can involve bilateral relationships such as loops between clusters and loops within the same clusters, namely interaction relationships between elements of the same clusters (inner dependencies) and/or elements of different clusters (outer dependencies). Fig. 1 shows these structural differences.
As a result, ANP can be useful in solving very complex MCDA problems in which inner and outer dependence relationships affect elements of the decision process. Moreover, its intuitive structure allows the deep involvement of all stakeholders and experts in the decision/evaluation process, so determining accurate and reliable prioritization results (Saaty & Vargas 2006). Actually, inner and outer dependence relationships and required pairwise comparisons are provided by a specifically selected decision group on the basis of opinions and viewpoints of its members. However, ANP in its typical formulation can be a cognitive demanding and a very time consuming task when a high number of pairwise comparisons need to be carried out. Moreover, it can imply equivocal questions to be answered, particularly as regards to inner dependencies. By integrating DEMATEL with ANP, the former is considered for dealing with inner dependencies. Actually, DEMATEL is based on cause-effect type relationships and thus constructing the direct relation matrices is a simple task for the involved respondents. In such a way, DEMATEL provides consistent information for the tackled problem and implicates a meaningful reduction of the pairwise comparisons number required by ANP. Hence, DANP-based models overcome the distinctive shortcomings in ANP analyses, and allow to successfully face with complex decision-making situations, as a consequence. Some relevant and recent studies describing these kind of MCDA models in very different research fields, in which qualitative aspects need to be analyzed in quantitative and reliable manner, are recapitulated in Table 1.  The DANP-based model considered in this study can be summarized in several fundamental phases as shown in Büyüközkan & Güleryüz 2016 (Fig. 2).
DEMATEL Phase -During this phase, several computational steps of DEMATEL are performed.
 Phase A.1 -Construct the initial direct relation matrix. Experts are asked to pairwise compare factors of each cluster in terms of influence and direction. These assessments are considered to construct the direct-relation matrix A. In such a matrix, the generic term ai,j identify the degree of influence of the factor i on the factor j, and the diagonal term ai,i is set to be equal to 0. To compare the relative influence degrees of factors, a comparison scale including the levels below reported is herein employed (Büyüközkan and Öztürkcan 2010): no influence (0), low influence (1), medium influence (2), high influence (3) and very high influence (4).  Phase A.4 -Calculate the factor's degrees of influence. After obtaining the total-relation matrix S, the influence of a factor on other factors and the extent to which it is influenced by other factors are calculated. Let Rj be the sum of the column j and Di the sum of row i of the total-relation matrix S: in which si,j is the generic element of the total-relation matrix S. Rj donates the total effects, both direct and indirect, received by the factor j from other factors, while Di indicates the total effects, both direct and indirect, given by the factor i to other factors.
 Phase A.5 -Draw the cause-effect diagram and construct the inner dependence matrix. The quantity (Dk + Rk) which is defined as the "prominence" gives an index representing the total effects both given and received by the factor k. In other words, it represents the degree of the importance (total sum of the effects given and received) that the factor k plays in the system. On the contrary, the quantity (Dk -Rk) which is defined as the "relation" shows the net effect that the factor k contributes in the system. When it is positive, the factor k is a net causer and when it is negative, the factor k is a net receiver A positive value of (Dk -Rk) indicates that the factor k is the cause criterion, while a negative value suggests that it is the effect criterion. By mapping the dataset of the (Dk + Rk); (Dk -Rk) k  , the cause-effect diagraph is obtained. In order to assure a suitable diagram representation, the experts involved need to set an influence threshold value, and only the influence levels in the matrix S greater in absolute value than such a value are shown into the cause-effect diagram. The last phase regards the construction of the inner dependence matrix V. Such a matrix is constructed by normalizing the total-relation matrix S. In the inner dependence matrix, the sum of each column has to be equal to 1.
ANP Phase -During this phase, several computational steps of ANP are performed.
 Phase B.1 -Construct the pairwise comparison matrix. The pairwise comparison matrix P is a squared reciprocal end positive matrix in which its generic term pi,j equal to 1 for i = j and pi,j = 1/pj,i for i ≠ j denotes the relative importance of the factor i against the factor j with respect to the goal. For the pairwise comparisons of factors, the well-known 9-point Saaty evaluation scale (Saaty 2008) is adopted. The involved experts valuate the relative dominance between each pair of compared factors as equally important, moderately more important, strongly more important, very strongly more important, and extremely more important. These judgments are then computed into crisp values of 1, 3, 5, 7, and 9, respectively. Reciprocals of these values are used for the corresponding transpose judgments.
 Phase B.2 -Calculate relative important weights. The row vector average method is used to normalize the results, and the approximate weight Wi is calculated via the Eq. (5) as below reported: The pairwise comparison matrix P completely responds to pi,k = pi,j·pj,k ∀ i, j, k. The following Eq. (6) is then applied to obtain the value of the largest eigenvalue λmax of P 3 -Check consistency of the pairwise comparison matrix P. The consistency index (CI) and consistency ratio (CR) are used to estimate the consistency of collected pairwise comparisons. These parameters are computed via Eqs. (7) and (8) below reported.
If CR is less than 0.1, the collected pairwise comparisons are considered as acceptable, otherwise the experts need to be again involved. RI is the average consistency index for randomly generated weights. When the number of compared elements n is equal to 2, 3,…,8, RI is equal to 0.00; 0.58; 0.90; 1.12; 1.24; 1.32; 1.41, respectively. Finally, the geometric mean is considered to aggregate the collected individual judgments to construct a single pairwise comparison matrix.  Phase B.4 -Construct a super-matrix by entering evaluations of experts. A super-matrix is defined as a partitioned matrix in which every sub-matrix consists of relationships between two clusters. Local priority vectors are considered in the corresponding columns in the super-matrix.
 Phase B.5 -The super-matrix is made stochastic, i.e. weighted super-matrix, where each column sums to 1. Then, the weighted super-matrix is raised to its limiting powers until the weights converge to stable values, thus forming the limit super-matrix.
 Phase B.6 -From the limit super-matrix, weights and priorities of factors concerning the considered alternatives are obtained.

Materials and methods
The MCDA methodological framework herein proposed is composed by three fundamental steps as shown in Fig. 3.

Fig. 3. Steps of the decision-making process
In the first step, the goal to be pursued is specified and the software alternatives to be analyzed are chosen. Subsequently, the faced decision-making problem is modelled based on options and viewpoints of a selected panel of experts. are involved in providing pairwise evaluations of factors with reference to the investigated alternatives. Weights and priorities of factors as concerns the considered alternatives close this step. Finally, a comparative validation study and a sensitivity analysis are conducted and obtained outcomes are critically stressed and discussed.

Problem formulation
As before said, it was carried out a usability analysis as concerns the two major MOSs (considered alternatives). More in detail, Android and IOS in their current versions, namely version 10 and version 10.3.4, respectively, were herein investigated. The two considered MOSs are herein anonymously referred as MOS1 and MOS2 to assume a position of independence as concerns obtained results, as well as emphasize how the proposed methodological framework works, its applicability and effectiveness in evaluating the software usability.
A panel of experts composed by 10 anonymous highly recognized and experienced professionals with several years of activity in the treated field and a strong knowledge about the two investigated MOSs was selected. More in detail, software designers and developers, software technicians, IOT security experts and an IT consultant, aged between 25 and 45, along with an academic in the quality management field, were the main professional specializations characterizing the involved panel members to assure their high consistency and authority (Murphy et al. 1998). Experts involved provided their opinions and viewpoints to select and evaluate the key elements of the treated MCDA problem (factors and clusters), thus, they were the main actors of the investigation herein performed.

Key usability factors and clusters of MOSs
First, a suitable structure of key factors and clusters as concerns the MOSs usability was described based on a comprehensive literature review as well as useful contributions from involved experts. This step was a necessary activity given the lack in the literature of studies concerning usability aspects of the MOSs. It began with the review of usability studies through which the most frequently used software usability aspects were identified. In the present paper, the following references were analyzed:  Han et al. (2000) developed an empirical usability model for consumer electronic products able to point out about the functional relationships among usability criteria and user interface aspects.  Calisir and Calisir (2004) developed a usability framework covering objective and subjective usability factors, this latter related to the user satisfaction such as perceived ease-of-use and perceived utility.  Seffah et al. (2006) unified existing usability standards and models into a consolidated hierarchical usability model which includes several usability attributes each of which corresponding to a specific usability aspect.  Singh and Wesson (2009) considered five usability criteria and a set of usability heuristics based on the Nielson's ten heuristics (Mack and Nielsen 1995) and Shneiderman's eight golden rules for the user interface design (Shneiderman & Plaisant 2010).  Coursaris and Kim (2011) developed an adapted usability evaluation framework to the context of the mobile computing environment. More in detail, the authors conducted a qualitative meta-analytical review of more than 100 empirical usability studies to point out about the key usability aspects for the investigated field.  Baharuddin et al. (2013) proposed a set of usability dimensions based on reviews of previous related studies.
Particularly, twenty-five dimensions were found out from the analysis which were synthesized and prioritized obtaining the ten most important usability dimensions. The usability factors identified are summarized in Table 2. Then, experts involved provided their opinions in order to select among identified usability factors those considered fundamental for the usability of MOSs. Therefore, identified usability factors were included in a questionnaire through which the experts were asked to evaluate their significance degree with relation to the MOSs usability. A 3-point evaluation scale was employed to assess individual judgments as essential, useful but not essential, or not necessary. The Content Validity Ratio (CVR) was considered to examine collected data, and with reference to the n th factor, the following relationship was employed: where ne is the number of experts indicating "essential" and N is the total number of experts involved. The threshold herein applied for CVRn is based on the evaluations carried out by Lawshe (1975). Thus, being the panel composed by 10 members a minimum CVRn of 0.62 is required to satisfy the five percent level of significance. Table 3 shows results obtained.  Finally, based on the experts' support the selected key usability factors were listed under the following 5 clusters: System quality (C1); Service quality (C2); Performance (C3); Effort (C4) and Acceptability (C5), as reported in Table 4. This structure of key usability factors and clusters was considered to describe the conceptual model of MOSs usability subsequently detailed. System quality (C1) refers to the key factors measuring the quality of a MOS considered as a system Navigability (F1,1) represents the degree to which the interface features of a MOS are well organized. Security (F1,2) measures the degree to which a MOS protects data from unauthorized uses. Flexibility (F1,3) refers to the MOS capability to allow upgrades required by the continuous mobile devices progress. Customizability (F1,4) namely the degree to which a MOS can be customized to meet users' needs and necessities.

Service quality (C2) refers to the key factors measuring quality of interaction between a MOS and users
Reliability (F2,1) refers to an extent to which a MOS is dependable, stable, and bug-free. Error prevention (F2,2) intended as the availability of downloadable applications to overcome possible operation gaps of a MOS. Responsiveness (F2,3) refers to an extent of the speed of the MOS's response to users' requests and actions. Performance (C3) comprises the key factors measuring quality of the tasks execution Efficiency (F3,1) which is an extent to which the execution of a task saves resource needed to complete such a task. Usefulness (F3,2) is an extent to which a MOS facilitates the user performance in the tasks execution. Effectiveness (F3,3) is the degree to which tasks can be executed accurately and completely by using a MOS.
Effort (C4) refers to the key factors that measure the intuitiveness of a MOS Accessibility (F4,1) which implies an extent to which a MOS can be used by people with the widest range of characteristics and capabilities. Controllability (F4,2) refers to a measure of the ease with which a MOS does what the user wants. Ease-of-use (F4,3) is an extent to which interaction with a MOS is free-of-effort. Understandability (F4,4) refers to an extent to which the interface functionalities of a MOS are clear and unambiguous. Acceptability (C5) refers to the key factors that measure likeability and behavioral intentions related to the MOS usage Satisfaction (F5,1) represents an extent to which the MOS use meets user's expectations and necessities. Aesthetics (F5,2) intended as the visual charm and pleasantness of the graphic interface of a MOS.

Decision network
The structure of the MOSs key usability factors and clusters summarized in Table 4 was taken into consideration at this step of the analysis. Specially, involved experts pointed out about the inner and outer dependencies between MOSs key usability factors and clusters, thus obtaining the decision network of the under taken MCDA problem. Particularly, it was hypothesized that the MOSs usability is affected by all the considered clusters, namely System quality (C1), Service quality (C2), Performance (C3), Effort (C4) and Acceptability (C5). Moreover, involved experts considered the following dependence relationships between clusters:  System quality (C1) affects Service quality (C2), Effort (C4) and Performance (C3);  Service quality (C2) affects Performance (C3) and Acceptability (C5);  Effort (C4) affects Performance (C3) and Acceptability (C5). Finally, also dependencies between elements in the same clusters were taken into account by experts. In such a way, the conceptual model of MOSs quality in terms of software usability shown in Fig. 4 was obtained.

MCDA investigation
Considering the complexity of the issues herein treated, at this step of the analysis experts were called to answer to questionnaires in consecutive rounds, and the following phases previously shown in Fig. 2 were carried out.
Phase A.1 -Experts provided the pairwise comparisons of the key usability factors for each cluster according to the 5-point scale of DEMATEL previously described, and the related direct relation matrices were constructed. As an example, considering System quality (C1), the initial direct relation matrix A was obtained by averaging the initial direct relation matrices provided by each involved expert (Table 5).

Table 5
Direct relation matrix A for System quality (C1)  (2) as shown in Table 6. In particular, K=1/7.6 = 0.131 and the generic term of M is obtained by multiplying K by the related term of the direct relation matrix A.
Phase A.3 -The total-relation matrix S was found out by using Eq.
(3) based on the normalized direct-relation matrix M. Such a matrix for System quality (C1) is shown in Table 7. Dk is the row sum and Rk the column sum.

Table 7
Total relation matrix S for System quality (C1) Phase A.4 -Values of (Rj) and (Di) were considered to obtain the prominence (Dk + Rk) and relation (Dk -Rk), as shown in Table 8.
Considering the usability factor k, (Dk + Rk) is the sum of the relationships among all the key usability factors and denotes the importance of such a factor. Moreover, if (Dk -Rk) > 0, then the usability factor k directly affects the other ones to a greater extent than it is affected by them.
Phase 2.5 -The impact-diagram map was constructed by using the dataset (Dk + Rk); (Dk -Rk) k  , and, in order to assure an appropriate diagram representation, the threshold was set by the experts to a value equal to 0.05 (Fig. 5).
Results obtained show that F1,4 and F1,3 are dispatchers while F1,2 and F1,1 are receivers. Moreover, from Table 8 it is possible to note that that the Customizability (F1,4) which has (Dk -Rk) = 0.613 is regarded as the most important cause since it influences all the other factors with a high importance level ((Dk + Rk) = 8.029). On the other hand, Security (F1,2) has (Dk -Rk) = -1.046 and presents the lowest importance ((Dk + Rk) = 4.751). Finally, the inner dependence matrix V1 was obtained based on total-relation matrix S (Table 9). For example, the value of 0.218 is obtained as 0.809/3.712 (see Table 7). The other usability clusters of MOSs were analogously examined by DEMATEL and their cause-effect diagrams (Fig. 6) as well as inner dependencies matrices were obtained (Tables 10 -13).

Table 10
Inner dependencies matrix V2 of the cluster (C2)

Table 12
Inner dependencies matrix V4 of the cluster (C4) Then, the ANP phases were performed.
Phase B.1 -The remaining nodes and alternatives of the decision network of Fig. 4 were analyzed by using ANP. Table 14 shows the notation employed for the sub-matrices of the super-matrix. The pairwise comparison matrices are indicated with letters, while "I" denotes the identity matrix. Moreover, V1, V2, V3, V4 and V5 are the inner dependence matrices previously estimated via DEMATEL.

Table 14
Sub-matrices notation for the super-matrix Based on outer dependencies depicted in the decision network shown in Fig. 4, the pairwise comparison matrices were constructed taking into account the experts' judgments expressed by the Saaty 1-9 scale.
Phase B.3 -The consistence of pairwise comparison matrices was checked by the consistency index (CI) and consistency ratio (CR) using Eqs. (7) and (8). Given that CR ≤ 0.10 for all the considered matrices, the consistency degree of collected pairwise comparisons was considered satisfactory. Finally, the collected individual judgments were aggregated via the geometric mean to construct a single comparison matrix. For System quality (C 1 ), Table 15 shows the obtained aggregated pair-vise comparison matrix B and related weights.  Table 4 were constructed and related weights were estimated.
Phase B.4 -Obtained weights for all the aggregated pairwise comparison matrices were placed in the corresponding columns of the super-matrix along with priorities calculated via DEMATEL, and the initial unweighted super-matrix was obtained.
Phase B.5 -The super-matrix was first made stochastic by normalizing to 1 the values of each column. Then, the latter was increased to sufficient large power until the weights convergence occurred. The obtained super-matrix was raised to its 5 th power and the resulting limit super-matrix was obtained.
Phase B.6 -From obtained limit super-matrix, weights and priorities of the key usability factors concerning the two investigated MOSs were obtained as reported in Table 16.

Results analysis and discussion
In order to evaluate the validity of outcomes obtained via the proposed DANP model, a comparative procedure based on ANP was performed (Azizi et al. 2014). Specially, only ANP in its typical formulation was employed to evaluate weights and priorities for each software alternative according to the identified decision network and evaluations provided by the involved panel of experts. In such a regards, they provided also pairwise comparisons as concerns inner dependencies required by the comparative procedure based on ANP. Obtained comparison results confirmed the validity of outcomes obtained in view of their substantial convergence with those obtained via the DANP model (Table A1 in Appendix 1). Moreover, a sensitivity analysis to assure robustness of the investigation outcomes was conducted. Such an analysis was performed by changing the value of the factors weights (Büyüközkan & Güleryüz 2016). The two more important usability factors, namely F5,1 and F2,1, were selected, and as a first case the weight of F5,1 was increased from 0.167 to 0.350 and weights of the other usability factors were changed proportionally. By following the same procedure, as a second case the weight of F2,1 was increased from 0.116 to 0.350. Results obtained are shown in Table A2 in the Appendix 1. In particular, there are not discrepancies in the alterative rankings considering all treated cases. Moreover, considering the alternatives priority values, ANP sensitivity results are slightly different from those obtained via the unaltered ANP analysis. This situation which does not characterize results of the DANP model, can denote an undesirable aspect since it may create pressures on experts involved in the decision process during the evaluation process.  (1) Obtained cause-effect diagrams of DEMATEL (Figs. 5 and 6) reveal directions and strengths of inner dependencies by providing useful information for an effective understanding of the MOSs usability concept. The causal relationships and obtained degrees of influence are described below.
These key usability factors slightly influence a few other key usability factors, indicating that they are relatively independent.  Weak relation, low prominence: Security (F1,2), Controllability (F4,2), Accessibility (F4,1). These key usability factors are slightly influenced by the other factors, suggesting that they are relatively independent.  Weak relation, high prominence: Efficiency (F3,1) and Error prevention (F2,2). These two key usability factors could not be directly improved being influenced by the other factors.
On the other hand, ANP results provide weights and priorities of the key usability factors with respect to the two investigated MOSs (Table 16). Weights are an importance measure of the key usability factors, and the greater scores are related to those ones which play a fundamental role in the human-computer interaction. Particularly, Satisfaction (F5,1), Reliability (F2,1) and Effectiveness (F3,3) were the usability factors resulting most important, while Accessibility (F4,1), Customizability (F1,4) and Aesthetics (F5,2) were the less important ones.
Regarding priorities, for each key usability factor the highest score is related to that MOS alternative which has better usability features concerning the considered key usability factor. More in detail, the alternative MOS2 was the best one in terms of usability with a global priority score of 0.613. Moreover, all key usability factors presented priority scores greater than those of MOS 1 and Aesthetics (F 5,2 ), Efficiency (F 3,1 ), Ease-of-use (F 4,3 ) and Understandability (F 4,4 ) were those best addressed. Obviously, considering MOS1, its best-addressed key usability factors are coincident with those less effective for MOS2 and vice versa. However, it is necessary highlight that the investigation herein carried out represents a preliminary study about MOSs usability. Actually, it is based on viewpoints of the involved experts. Moreover, the latter were selected by following opportunity criteria to assure consistence and reliability of outcomes from a professional and qualified perspective but also quickness in performing the data collection phase. Actually, the intense cognitive load arising from the numerous answers required by the performed comparative validation procedure based on the typical ANP did not allow extending such a study to a larger sample of experts.

Conclusions
The present paper suggested a methodological framework based on a MCDA model integrating DEMATEL along with ANP for evaluating software usability. DEMATEL was concerned about pairwise influences between usability factors and allowed to quantify the related cause-effect relationships. Conversely, ANP was employed as concerns outer dependencies describing the conceptual model of MOSs usability. In such a way, a meaningful reduction of both cognitive complexity and number of the pairwise comparisons required as regards to the typical ANP were achieved without affecting quality of obtained key outcomes. These aspects make the proposed MCDA model undemanding and highly applicable, as a consequence. To show how the proposed framework works, a preliminary usability investigation as concerns the two major Mobile Operating Systems (MOSs) was carried out taking into consideration viewpoints and opinions of a selected highly experienced panel of experts. A comparative analysis was also carried out to test the validity of results obtained and a sensitivity study was conducted to assure their robustness. Results obtained pointed out about crucial aspects for driving MOSs usability improvement processes and strategies towards the addressing of the highlighted discrepancies in usability, thus demonstrating the effectiveness of the proposed methodological framework.
Main recommendations for future developments of the proposed framework concern the accessibility of its mathematical foundation and consistency of its outcomes, as well the possibility of combining it with other suitable MCDA methods. The use of the Fuzzy Set Theory in the evaluation process allows handling the potential epistemic uncertainty of experts involved in the study. Moreover, a further possible future development could be characterized by the combined use of other MCDA methods such as VIKOR and TOPSIS (Opricovic & Tzeng 2004) to support the evaluation process of the decision network elements. Finally, being the weighting process performed by involved experts based on their personal viewpoints and perspectives, such an aspect might potentially, consciously or unintentionally, lead to biased allocations of weightings. Thus, some caution should be given in interpreting obtained outcomes and for the related generalization. For this reason, broadened analyses might be carried out in the future in the under analysis context to confirm outcomes herein obtained. In addition, studies about other software quality aspects might be developed via the proposed methodological framework, so as to perform further tests and draw out detailed comparisons.
In this section, results obtained via the conducted validity and sensitivity analyses are reported.