Usability evaluation of e-government websites: A case study from Taiwan

In the Information and Communications Technology era, eGovernment projects present a great opportunity for governments to offer better and quicker services to their users from the public. However, the success and the failure of these projects to achieve the expected goals depend heav- ily on some important aspects; mainly websites usability. This study intends to evaluate usability of current e-government websites in Taiwan. The results indicate that a number of usability prob- lems have been found in the target e-government websites. These findings can help designers identify users’ usability requirements and draw their particular attention to further develop more usable e-government websites.

In the Information and Communications Technology era, eGovernment projects present a great opportunity for governments to offer better and quicker services to their users from the public. However, the success and the failure of these projects to achieve the expected goals depend heavily on some important aspects; mainly websites usability. This study intends to evaluate usability of current e-government websites in Taiwan. The results indicate that a number of usability problems have been found in the target e-government websites. These findings can help designers identify users' usability requirements and draw their particular attention to further develop more usable e-government websites.

Introduction
Information and Communication Technology (ICT) has made government services more efficient and effective to its populace. Furthermore, it has resulted in domestic and international usage of services (Kumar et al., 2007). One of the most exceptional initiatives of the ICT is the launching of Electronic Government ("eGovernment"). The eGovernment services that many governments around the world present to the general public has significantly enhanced governmental services in effectiveness and efficiency therefore improving economic and social development both domestically and internationally (Kumar et al., 2007). Upgrading and enhancing the conventional and traditional way government functions was a consequence of the introduction of web based eGovernment which resulted in 95% of UN member states utilizing it (UN Public Administration Programme, 2010), because it has also improved accessibility and performance of government. The development of such systems increased access and distribution of governmental services. Internet and computer access are not the only variables to consider when aspiring to improve access and distribution, the governmental and populace perspectives or experiences, and any gaps therein (Heeks, 2003;Choudrie et al., 2009), should be considered. This has exposed a lack of an individual oriented needs perspective that usability focuses on. Usability became important as public agencies attempted to improve access and cost cutting measures (Brown & Brudney, 2004). Several investigations involving eGovernment services explain usability as the fundamental trigger for the projects' adoption (Bwalya, 2009;Kumar et al., 2007). Through the presentation of eGovernment and much improved usability functions, civil service efficiency can be enhanced. Additionally, an individual's satisfaction can improve as boosting one's engagement with eGovernment services is encouraged. Based on Casaló et al. (2005), failing to accomplish appropriate degrees of usability for eGovernment services sabotages not only the eGovernment endeavors, but also the connection between the government and individuals. Usability is considered a significantly essential element in measuring the effectiveness of eGovernment projects. A general standard of usability needs to be obvious to the individual using eGovernment sites. Therefore, any eGovernment project should pay particular and adequate attention in its plan for developing a website, as this is the foundational factor for the project's effectiveness. Regarding to Anderson and Henriksen (2005), "whilst few could complain about the amount of information on government sites, [far too] many have [repeatedly] complained about the ease with which users find information".
The paper is structured as follows: section 2 reviews relevant studies to indicate the importance of usability to e-government websites. This is followed by designing an empirical study to evaluate usability (section 3). Section 4 discusses the evaluation results. Finally, the conclusion and future research are suggested in (section 5).

E-government
The e-government websites have developed and catered to different foci such as organizational, functional, governmental, and business. This evolution has displayed the current development and future areas of improvement (Tambouris et al., 2001). Today, e government can make substantial efforts to provide their services to residents, enterprises, and other government companies through the Internet (Tambouris et al., 2001). Additionally, prospective eGovernment objectives are shifting from nationwide to regional (West, 2005). Domestic national governments have been introducing newer technology and have led the way for more localized government agencies to follow (Gil- Garcia, 2006Garcia, , 2007Gil-Garcia and Martinez-Moyano, 2007). The obvious advantages of e-government are that it can boost and enhance service delivery (Mutula & Wamukoya, 2007); transparency (Ciborra, 2005); civil service overall performance (Kumar et al., 2007); policy effectiveness (OECD, 2003); reinforce citizen trust (Eyob, 2004) and accomplish massive cost savings (Culbertson, 2002).

E Government development
The evolution in IT (information technology) and conventional governmental needs have strongly fueled the advancement of e-government (Strejcek and Theil, 2002;Torres et al., 2005). Conventional governmental services have been highly complicated with bureaucratic elements (Cairns et al., 2004) making access to information unnecessarily hard and supply of services exasperating and tiresome for individuals (Cairns et al., 2004). These barriers have caused negative suspicion of government services, which consequently resulted in low public involvement and a reduced confidence in government (Eyob, 2004). The common expectations of individuals were to see the implementation of a newer and fresher methodology that enables better access whilst providing a richer source of pertinent information (OECD, 2003). Therefore, government needs to modify the administration and processing of official governmental affairs (Yang et al., 2005), providing superior quality and enhanced user-friendly governmental services (Barnes & Vidgen, 2004), thus producing increased ease and frequency of participation (Kumar et al., 2007). Based on these criteria, e government will become an exceptional channel to achieve these objectives (Kelly & Tastle, 2004;Metaxiotis & Psarras, 2004;OECD, 2003). Currently of the 192 UN member states almost 98% of them have built their web-based e-government systems (UN Public Administration Programme, 2010).

eGovernment in Taiwan
Internet based innovation is something Taiwan has commonly been known for and specifically the advancement of e government technology and services. In the 1980s the Taiwanese government pursued a strategy to automate and computerize the general public sector to enhance operational processes, service efficiency and quality, cost cutting measures, and the provision of innovative and easy services to the private sector and to its citizens (Lee, 2003). The Taiwanese government also introduced the National Information Infrastructure (NII), which was targeted at building national ICT and establishing Taiwan as a telecommunications hub in the Asia-Pacific region (Wang, 1999). E-government websites developed by the Taiwanese government offered the provision of information and innovative services such as eprocurement and e-taxation. The 'E-government Entry Point of Taiwan' (http://www.gov.tw) is a centralized government portal that combines all online government services and information. Electronic certificates, payment methods, form generators, and work flow utility systems used to accelerate the advancement of online software services which are the fundamental features of this portal. By October 2001 Taiwan was second out of 196 countries on its improvement in the direction of government digitalization, (WMRC, 2001). It was first, just two years later, when it came to information availability, resident access, portal access, databases, and services delivery (West, 2006). On November 24, 2016, the Executive Yuan, Taiwan launched the Digital Nation and Innovative Economic Development Plan (2017-2025) (DIGI+ program). The plan's main goals for 2025 are to grow Taiwan's digital economy to US$205.9 billion, increase the digital lifestyle services penetration rate to 80 percent, speed up broadband connections to 2 Gbps, ensure citizens' fundamental rights to have 25 Mbps broadband access, and put Taiwan among the top 10 information technology nations worldwide. Accordingly, the national development council (NDC) of Taiwan promoted the national Digital Government program of Taiwan (2017-2020). In 2017, as the fifth stage of e-Government development in Taiwan. The newly launched program aims to adopt emerging technologies such as big data, cloud computing, artificial intelligence, etc. to build a comprehensive global leading e-government of which citizens can take full advantage instead of receiving standard public services. Data-driven policy-making, citizen-centric service, and public-private participation are the three main core concepts of the program.

Usability
Usability is a well-known principle in Human-Computer Interaction research and it determines how simple and effective it is for any individual to carry out duties when making use of a product (Han et al., 2001). Research reveals that usability is an essential element in identifying effectiveness (Bevan, 1995;Karahoca et al., 2010;Park & Lim, 1999), and frequency of individuals' engagement (Lee & Koubek, 2010;Sauer and Sonderegger, 2010). Consequently, usability has been broadly mentioned in products, projects, and system design. Extant usability literature in the context of e-government websites has mostly focused on two well-formulated sets of heuristics, namely Nielsen's usability heuristics (Nielsen & Molich, 1990;Nielsen, 1994) and the six-dimensional framework (Baker, 2009). The Nielsen heuristics initially developed by Nielsen and Molich (1990), and further refined by Nielsen (1994), have been widely published and used for usability evaluation. These heuristics consist of 10 items that were primarily developed for the usability evaluation of user interfaces. These items include the visibility of system status, match between system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, help users re-cognize, diagnose, and recover from errors, and help and documentation (Nielsen, 1994).

Usability effects on e-government users
The usability of e-government has consequences and effect on individuals' satisfaction, expectation and belief. Magoutas and Mentzas's (2010) research measured the level of individuals' satisfaction with eGovernment services. The factors indicative of individuals by interaction of forms, website usability, security, information quality, service reliability and support mechanisms. Verdegem and Verleye (2009), investigated numerous individuals' expectations about e-government. A sample of 5590 participants, revealed that individual choices is dependent on usability such as accessibility of e-government services, ease of finding the e government website, the loading speed of the website, the usefulness of information offered on the website and the overall flexibility. Increased degree of usability improves the individuals' expectation also affecting the lasting impact on use of e-government web services. Kumar et al. (2007) showed identical outcomes and recommended a conceptual model to analyze and assess the impact of individuals' adoption of e-government, such as website design, service quality, individuals' satisfaction and characteristics. The outcomes revealed that website design, such as effectiveness and efficiency of use, behaves as the critical component, not only impacts individuals' experience, but also leave individuals' interaction with e government positively. Hence, they recommended that enhancing website design, specifically with regards to navigation, aesthetics, content, accessibility, and customization is extremely likely to motivate users' adoption of e government. In closing, this compilation of research showed the importance of usability to e government, and its impact on individuals' mindset, perception, and use. Furthermore, the results show that the critical component to accomplishing e government's objectives is usability and that addressing it is imperative when developing e government.

Methodology
To conduct the research, this study applies a quasi-experimental study as the research method to collect data through the questionnaire. Experimental study is typically used to measure "cause and effect" relationship under controlled conditions and environments. Within an experimental study, one situation can be altered by bringing an extraneous variable into it. Each situation can be re-evaluated after the intervening alteration. The changes in re-evaluation can be caused by the extraneous variable. This feature is particularly suitable for the purpose of this study because this study aims to evaluate the usability of the e-government websites, identifying the usability problems. To carry out the evaluation, a quantitative approach is applied to the study. The evaluation consists of the heuristic evaluation. To collect research data, the questionnaire technique is employed in the study. This research technique can be used to approach the research questions from different aspects. More specifically, the questionnaire aims to capture users" perception to assess the usability of the e-government websites. Finally, the specific questions are developed, based on these heuristic criteria (See Fig. 1 and Fig. 2).  Table 1 Explanations of Nielsen's usability heuristics evaluation (Nielsen, 1994) Usability Heuristics Explanations H1. Visibility (V) The site should keep users informed about what is going on through appropriate feedback within a reasonable time. H2. Match between system and the real world (MR) The site should use the user" language, follow real-world conventions, make information appeared in a natural and logical order.

H3. User control and freedom (F)
The site should make undo and redo functions available during interaction and support users to leave the site at all times. H4. Consistency and Standards (CS) The site should keep the same design features and follow platform conventions through the site.

H5. Error prevention
The site should support users to overcome errors and prevent the same problem occurrence.

H6. Recognition (R)
The site should make objects, actions and options easy to remember. In addition, instruments on the site should be visible and easily retrievable.

H7. Flexibility and efficiency of use (FE)
The site should consider the usage for both novice users and experienced users. Furthermore, it allows users to tailor frequent actions.

H8. Aesthetic and minimalist design (AM)
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. H9. Help users recognize, diagnose, and recover from errors (HU) The site should indicate error messages. Error messages should precisely indicate the problem and constructively suggest a solution.

H10. Help and documentation (H)
The site should provide help and documentation that can be easy to search, focus on the users" tasks, list concrete steps to support users.

The selection of e government websites
The local level of e-government website is selected in this study, because 1) it is the closest level for users; 2) it is often used by the public since the local level is more informational for users and focuses on the needs of users in accessing information and services (Reddick, 2009); 3) it can show the effects of egovernment on users (Tolbert and Mossberger, 2003) and 4) evidence from studies finds that bigger challenges exist at the local level of e-governments and in their website design. Therefore, the study uses two local e government websites in Taiwan: called national taxation, ministry of finance (web1), and ministry of the interior (web 2).

Evaluation Process
To conduct the evaluation, 100 participants were assigned to evaluate two target e government websites.
Each target e-government website evaluation involved 50 participants. Each participant follows the same evaluation process, which are: free review, and completing the questionnaire. Free review allows the participants to look through the target e-government website several times. They can freely either look at the overall e-government website or focus on the specific website design elements. After reviewing the target websites, the participants are finally asked to fill in the usability questionnaire.

Data Analysis
The data analysis techniques used are 1) The one-way ANOVA and the one-sample T-test. Statistical analysis is conducted using SPSS for windows (version 25). The significant value (P) is pre-defined as less than 0.05. In detail, to indicate whether the web1 and web2 have a difference in the participants' overall perception of usability, a one-way ANOVA is conducted with web1 and web2 as independent variables and usability perception as the dependent variable. Similarly, the one-way ANOVA is employed to indicate whether the web1 and web2 show differences in users' performance. In addition, to indicate whether there is a difference between the perception of overall usability and the perception of specific usability features in each target web1 and web2, a one-sample T-test is conducted.

2) Partial Least Square (PLS) which is a variant based alternative method of Structural Equation
Modeling (SEM) method. The Partial Least Square (PLS) analysis technique is done in three stages, namely: 1) The first step is to carry out an outer model test (measurement model), which is to test the validity and reliability of the constructs of each indicator.
2) The second stage is to test the structural model, carried out to ensure that the structural model built is robotic and accurate. 3) Hypothesis Testing. Hypothesis testing is done by looking at the probability value and t-statistics.

Validity test
Validity indicators can be seen from the value convergent validity. The value of convergent validity is the loading value of a factor in the latent variable with indicators. The value of Convergent validity can be seen in Table 2. In the model PLS, indicators can be a valid if it has loading value 0.5 -0.6. Convergent validity explains the ability of each indicator to explain the research variables studied. The latent variable indicator should be greater than 0.5. Indicates that all indicator results of each variable have the value of loading factor more than 0.5 then each indicator is declared valid. Testing the validity of an indicator, not only seen from the value of loading factor but can be seen also from the value convergent validity. Convergent Validity is a test conducted to see the validity of each indicator by looking at the value of the Average Variance Extracted (AVE) of each variable where the value must be greater than 0.5 so that it can be said to have value Good validity convergent.
Validity indicators can also be seen from the value of loading factor. When the value of loading factor of an indicator is more than 0.5 then it can be said to be valid. Conversely, when the loading factor value is less than 0.5 then it is removed from the model.   Table 3 describes the value of AVE from visibility, mapping, freedom, recognition, error prevention, consistency, error recovery, minimalist, help, and flexibility variables that can be seen that each of those variables has an AVE value above 0.5.

Reliability Test
Composite Reliability is a test conducted to see the reliability of each variable indicator. A data is said to be reliable if the composite reliability value is more than 0.7. From Table 4 this indicates that the internal consistency of the dependent entire variable have a good reliability value.

Users' perspective
In general, the results indicate that usability has been considered in the two e government websites as a number of usability strengths have been detected (Table 5). However, a number of usability problems have also been identified in each target e-government website (see Table 6). Among these problems, a lower mean score indicates a more serious problem. In web1, the most serious usability problem is that users are having difficulties to find out which links/subjects matched to the information they want. In web 2, the most serious usability problem is that users are confused by links that have many different colors. Link color is used to indicate different resources within the site. An appropriate number of link colors can visually support users to distinguish between resource differences, so that target information can be easily located to meet users' needs. Using this, users with limited color vision can quickly recognize the difference among subjects (Kappel et al., 2006). On the contrary, links with many different colors may visually influence the site's appearance and obstruct users' color vision, which may result in difficulty and confusion in information identification in web 2. After using one way ANOVA Test, we note in Table 7 that significance values (sig.) for V (visibility), MR (Match between system and the real world), R(Recognition rahetr than recall), AM(Aesthetic and minimalist design), HU (Help user recognize, diagnose, and recover from errors), H(Help and documentation) are less than (0.05), while significance value (sig.) for CS(Consistency and standards), EP(Error prevention), EF(Flexibility and efficiency of use) F(User control and freedom) more than (0.05). So we can conclude that there are statistical differences in terms of (visibility, Match between system and the real world, Recognition rahetr than recall, Aesthetic and minimalist design, Help user recognize, diagnose, and recover from errors, and Help and documentation) of the existing e government websites in Taiwan. However, there is no statistical difference in terms of (Consistency and standards, Error prevention, Flexibility and efficiency of use, User control and freedom) within the tested websites.

Hypothesis Test
The hypothesis testing is based on the significance value in testing the structural model that can be seen from the T-Statistic value between the independent variables and the dependent variables in the Path Coefficient table. For hypothesis testing in this study used an alpha value of 5% with a T-  Table coefficient path and T-Test can be seen in Table 8. Based on path coefficient matriks above, obtained T statistical value for all hypothesis, we can conclude that only HU (Help user recognize, diagnose, and recover from errors), and F (User control and freedom), EF (Flexibility and efficiency of use), V (Visibility System Status), have significant effect to ETU (Easy to Use). In contrast, other factors have not statistically significant impact on the ETU (easy to use). Then, these findings support the works of Verdegem and Verleye (2009) that emphasizes the felxibility of use as the important factor in ease to use of eGovernment. These findings also provides the suggestion to the Goverment of Taiwan to make an improvement on other factor including,i.e: EP (Error prevention), H (Help and documentation), CS (Consistency and standards), AM (Aesthetic and minimalist design), R (Recognition rahetr than recall), MR (Match between system and the real world), . The improvement of these factors is in line with the reccomendation of Kumar et al. (2007) that suggest enhancing website design, specifically with regards to navigation, aesthetics, content, accessibility, and customization is extremely likely to motivate users' adoption of e government.

Conclusions
This study has evaluated the usability of current e-government websites in Taiwan. A number of usability problems have been found in the target e-governments. This suggests that current e-governments need to improve their usability. These results can help designers understand e-government websites usability, especially identifying users' usability requirements. Furthermore, these identified usability problems can draw designers' particular attention to their websites and support them to further improve usability. Furthermore, the common usability problems detected can be used as reference to check usability for other e-government websites. However, this study has some limitations. For example, regarding usability criteria development, some criteria might be found to relate to more than one heuristic; however, the study grouped these criteria into one heuristic based on their key features. Moreover, to fulfill a thorough usability study, there is a need to offer concrete prescription for the Identified usability problems in order to improve usability of the target e-government websites. Future research, evaluates more e government websites from locals' perspective and especially from foreigners' perspective, and gives suggestions and solutions to the e government to how to improve their websites.