Extending and Refining Usability Heuristics to Better Address User Centered Design Issues in the Development and Evaluation of Municipal Websites

Usability of public administration websites is a key issue in the information society. Unfortunately, the web content is still difficult to use if not unusable in many websites. Developers seem to neglect basic principles of user centered design. Usability heuristics are valuable resources for both developers and evaluators during and after the development process. The purpose of this work is to extend and refine an existing set of heuristics in order to better address some user centered design issues. Previous studies revealed specific usability problems of municipal websites as well as some evaluation issues. Two heuristics were added that refer to user guidance and task support. The revised heuristics could better help evaluators to explain and developers to understand usability problems in municipal websites.


Introduction
The information society requires good quality information and better access to egovernment services for end users.In this respect, usability of public administration websites is a key issue.According to many authors, effective e-government services for the citizen means useful and usable applications [2], [19], [30].Unfortunately, the web content is still difficult to use if not unusable in many municipal websites.The developers seem to neglect the basic principles of user centered design.The websites are more oriented towards the presentation of the town, City Hall structure, and offices than to the practical information a citizen might need to solve a concrete problem.User centered design requires understanding users and tasks.This means to know who the users are and which are the goals they try to achieve.It also means to take a task-based approach to design and evaluation.Improving website usability requires usability heuristics and guidelines that could better orient development and evaluation.Usability heuristics play an important role in the design process and should be both useful and usable for developers and evaluators.Up to now there are two kinds of approach to website evaluation and two kinds of evalua-tion criteria that are used to evaluate municipal websites for usability.First approach aims at assessing website usability at general level.The evaluation criteria refer to general usability measures that are coded on some scale (e.g. a four-point scale [12]).The evaluation attempts to measure the degree to which such criteria are respected.Based on such measures a usability score is computed that enables ranking and comparing of municipal websites.The second approach aims at identifying the difficulties a user has in performing a task and achieving a task goal with the website.The evaluation criteria refer to various design knowledge structured in form of principles, criteria, heuristics, and / or guidelines.This enables evaluators to identify and explain usability problems.While the first approach to evaluation is summative the second is formative since it helps to uncover and fix usability problems as early as possible during the development process [32].Although both approaches are useful, the latter is closer to end user requirements and could substantially improve the usability of public websites.Previous usability studies targeting municipal websites revealed several specific usability problems [15], [28], [29].Usability evalua-1 DOI: 10.12948/issn14531305/18.1.2014.07tion has been carried out in a scenario-based approach and revealed some difficulties in using an existing set of usability heuristics to explain and document these specific problems.In this paper an extended and refined set of usability heuristics is proposed that elaborates on some insights gained from previous usability studies.In this respect, the existing set is extended with two usability heuristics related to user guidance and task support.The rest of this paper is organized as follows.
In the next section we present related work in usability evaluation with a focus on local egovernment websites.In section 3 we present the revised set of usability heuristics.The paper ends with conclusion and future work in section 4.

User-Centered Design Issues
Since the adoption of the framework of ISO standard 13407 [16] on user centered design, many researchers advocated for usercentered and a task-based approach to design and evaluation.This requires taking an iterative approach to development, active participation of users, understanding tasks and goals, and evaluation in context.Gulliksen et al. [11] proposed a set of design principles for user-centered design.They recommend creating a user centered attitude throughout the project team and evaluate the use in context.In order to produce usable applications usability guidelines should control the development.Kamper [18] noticed that the current usability practices are more oriented towards fault finding instead of goal achievement.Most usability heuristics are useful to identify usability problems in the design but less useful to measure the degree to which the task goal is accomplished.He proposed a set of 18 heuristics grouped under three general principles: guidance to successful achievement of goals, support during interaction, and increase of tasks' effectiveness and efficiency.Cockton et al. [5] argued that heuristic evaluation could serve as a starting point for user testing.In this respect, tasks for usability testing could be used to validate the problems anticipated by experts.In a recent web usability column, McCloskey [21] advocated for turning user goals into task scenarios able to motivate the user during usability testing.The scenario of use should be realistic, putting tasks in context in order to engage the users.Jong & Lentz [17] argued for a scenariobased evaluation of municipal websites in order to compensate some weaknesses of heuristic evaluation.In their approach, the evaluator expertise becomes less important and the reviewer has to test the website and rate its usability according to the task goal.

Usability Evaluation Methods
Usability evaluation aims at finding and reporting usability problems.The ultimate goal of usability evaluation is to help developers to fix the problems and improve the usability of the interactive system.A broader goal is to create a user centered attitude, provide good practice, useful design knowledge and case studies.Nielsen [24] defined a usability problem as any aspect of the user interface which might create difficulties to the user with respect to an important usability indicator (such as: ease to understanding, ease of learning, ease of use, and subjective user satisfaction).According to the potential effect on user's activities usability problems are ranked as severe, moderate and minor problems.Ranking is done according to the user's task.A severe usability problem means that s (he) is not able to accomplish the task goal or the task ends with a significant loss of data or time.The problem is moderate if it has an important impact on task execution but the user is able to find a solution.A minor usability problem is irritating the user but it doesn't have an important impact on accomplishing the task's goal.Prior to the first release of the website, all important usability problems (severe and moderate) should be fixed.Usability evaluation methods could be classified in two broad categories: inspection DOI: 10.12948/issn14531305/18.1.2014.07methods (faster and cheaper) [5] and user testing (expensive) [8].Reliability and validity of individual usability evaluation methods is a subject of debate in recent years [14], [22].Usability inspection (also termed as expert evaluation) is done by experts that are evaluating the user interface against a set of broadly accepted principles.Regardless the method used usability problems have to be described in detail, clearly explained and documented.An evaluation report should be both reliable and useful for developers [23].Usability evaluation is done in two phases: individual evaluation and consolidation.In the consolidation phase the usability problems identified by each evaluator are analyzed and merged to produce a list of unique usability problems.Several techniques for matching usability problems exist [13]: similar changes (fixing the problems leads to similar changes in the application), practical prioritization (prioritized list of usability findings), matching of description components (cause, breakdown, outcome, and design change), and problem type (classification of problems).Each technique leads to a different grouping of usability problems and number of single problems agreed [13].

Public Websites Usability
Donker-Kuijer et al. [7] comparatively analyzed five sets of e-government heuristics: guide for UK government websites, guide for Dutch government websites, top ten guidelines promoted by EU, guide of German government agencies, and Section 508 regarding accessibility (USA).All five sets have a strong focus on accessibility.Also, all are complex documents so the authors of the study have doubts regarding their usefulness for experts.Golzer & Kim [12] investigated the digital governance throughout the world in 2003, 2005 and 2007 and published a longitudinal study.They used a sample of 86 cities (country capital) from which 36 are European municipalities.The evaluation instrument has five components equally weighted: privacy / security, usability, content, services, and citizen participation.Usability was measured with 20 indicators focusing on the following aspects: user-friendly design, branding, length of homepage, targeted audience links or channels, and site search capabilities.Baker [2] argued for increasing the effectiveness of e-government through usability benchmarks.His methodological approach is based on six usability dimensions: online services, user-help, navigation, legitimacy, information architecture, and accessibility.The dimensions are equally weighted and for each of them several variables (measures) are considered.These variables are the basis for computing additive scores and to assess the overall usability score and usability scores by dimension.Sorum et al. [30] investigated the measurement of website quality and user satisfaction in Nordic countries.The results show that there is no positive correlation between these two indicators which could be explained by a mismatch between the quality criteria (highly technical) and traditional usability focus on users.Their recommendation was to pay more attention to users' needs and expectations.Barnes & Vidgen [3] evaluated an egovernment service for tax payment with the eQual questionnaire.eQual is measuring the perceived quality of the web service and has 23 questions grouped into three categories: usability, information quality, and service interaction.Users' comments were analyzed and the quantitative and qualitative results were triangulated.A finding of their research was that users are more critical as regarding usability when using comments.Detlor et al. [6] analyzed the influence of information quality on the use of municipality portals in Canada.They found that information quality has a strong indirect effect on the use and direct effect on the perceived ease of use.Although website usability is a key concern for an effective e-government, there are relatively few studies targeting the usability of municipal websites (i.e.local public admin-DOI: 10.12948/issn14531305/18.1.2014.07istration).Moreover, there are even fewer papers reporting detailed evaluation results.In a recent study the websites usability of municipalities in Alabama was investigated [33].The evaluation targeted only the home page and authors used a heuristic-based content analysis.Among the negative aspects, the authors mentioned: lack of breadcrumb trails, lack of city logo as a link to home page, lack of search capabilities, and problems with link color.

Usability of Romanian Websites
Usability of public websites is not yet a concern for developers in Romania.Several recent usability studies have been carried on in the context of a national research grant funded by the Ministry of Communication and Information Society (MCSI).The evaluation has been done in a scenario-based approach and the results revealed many usability problems.The first study targeted the municipal websites of four important towns (county capitals) [28].The evaluation method was usability inspection (four evaluators).Each evaluator tested the usability against three tasks: (T1) to know where and how register for audience, (T2) to identify and download the forms needed to get a birth certificate for a child and benefit from the state allowance and also where to send the application, and (T3) to find the date of the next Local Council meeting and the contact person for getting informed on the agenda and also to find and read the minute of the last meeting held in 2009.A total of 30 important usability problems were identified (from which 10 were critical).Critical problems were related to missing links, lack of information as regarding the date and agenda of the next Local Council meeting.The second study targeted the website of an important town (county capital) [15].Usability was tested against two tasks: (T1) checking the availability of online documents for local taxes and (T2) registration of a new client for online payment.Two evaluation methods were used: usability inspection and user testing (with think aloud protocol).The usability inspection anticipated a total of 14 important usability problems (5 critical).User testing confirmed 10 problems (from which 2 were critical).Most of them were related to error correction and user guidance.In this study only one website was evaluated since the main purpose was to test the methodology and to check the reliability and validity of usability inspection results.The third study targeted the websites of five City Halls of Bucharest sectors [29].A usability inspection was carried on with three evaluators that tested the usability against two tasks: (T1) get informed (procedure, required documents, and contact information) and download online documents for registration of a marriage and (T2) get informed and download online documents for changing an expired id card.A total of 32 important usability problems were identified (from which were 3 critical).Critical problems were related to the lack of online documents, impossibility of downloading, and many difficulties in getting oriented on the website.

A Revised Set of Usability Heuristics 3.1 Usability Heuristics
Heuristics are widely recognized principles.Nielson defined the heuristic evaluation as a usability inspection method based testing the user interface with ten heuristics [24]: visibility of system status, compatibility with the activity, user freedom and control, consistency, error prevention, recognition instead of recall, flexibility, aesthetics and minimalist design, and quality of error messages.Another approach in structuring design knowledge was proposed by Bastien & Scapin [4] as a set ergonomic criteria consisting of 18 elementary criteria grouped into 8 categories (general principles).Evaluating the interface with a small set of principles requires skilled experts in order to identify most of the usability problems.There are many approaches to extending the set of principles in order to address more specialized systems.Bach & Scapin [1] adapted the ergonomic criteria to virtual environments, such as VR (Virtual Reality) or AR (Augmented Reali-DOI: 10.12948/issn14531305/18.1.2014.07ty).One criterion was modified and two new criteria were added to the set in order to address specific interaction issues.Their approach was based on integrating the design knowledge (principles and guidelines) which is relevant for the target domain in the existing ergonomic criteria.Sutcliffe & Gault [31] proposed 12 heuristics derived from Nielsen's set and their previous work on VR: natural engagement, compatibility with user's task and domain, natural expression of action, close coordination of action and representation, realistic feedback, faithful viewpoints, navigation and orientation support, clear entry and exit points, consistent departures, support for learning, clear turn-taking, and sense of presence.Pinelle et al. [26] developed a set of heuristics for video game design.They based their approach on an analysis of usability problems that are relevant for current games.The relevance was weighted according to the following specific usability aspects: learning, controlling, and understanding a game.They proposed ten usability heuristics: consistent response to user actions, customization (video / audio settings, difficulty, game speed), predictable and reasonable behavior, unobstructed views (appropriate for current action), skipping non-playable and repeated content, intuitive and customizable input mappings, easy to manage controls, information on game status, instructions & help, and easy to interpret visual representations.Gorgan et al. [10] proposed a methodology supporting the development of new heuristics.The methodology was experimented on a set of specific usability heuristics for parallel and distributed applications.Gonzales et al. [9] extended the heuristics set with four heuristics that are specific for websites: search, news, updated information, and information architecture.They also detailed the 14 heuristics with several sub-heuristics.Ling & Salvendy [20] discussed in more detail the strengths and weaknesses of heuristic evaluation.Their work provides several references regarding extensions of the method and extension / specialization of the heuristics.

Existing Set of Usability Heuristics
Several authors questioned the effectiveness of heuristics in finding usability problems [18], [20], [25].In our opinion heuristics are valuable design knowledge but their effectiveness is mainly related to help developers on how to avoid and help evaluators on how to explain usability problems.In order to effectively identify usability problems a taskbased approach to evaluation is needed.In our previous studies we used a set of 24 heuristics which are grouped into six ergonomic criteria:  User guidance  Work load  Adaptability and control. Error management  Consistency and standards  Compatibility The set has been created by integrating the ergonomic criteria of Bastien & Scapin [4] with the heuristics of Nielsen [24].Since both original sources have been validated in several studies we considered that they are both reliable and useful.The heuristics were incorporated into a tool (software assistant) that enables a computeraided approach to evaluation [27].

Fig. 1. Editing a usability problem
The evaluator is able to specify the tasks, describe each usability problem, assign heuristics, and consult guidelines.Usability problems are described in a separate dialog unit (see Figure 1).The description includes: context, difficulties, cause, suggestions, and se-DOI: 10.12948/issn14531305/18.1.2014.07verity.For each selected heuristic explanations are given in a text box (right-down corner) and several guidelines are available ("Show guidelines" button) so the evaluator could go into more detail when documenting a usability problem.Heuristics and associated guidelines are used to explain and document each usability problem as well as to train evaluators in carrying on a usability inspection.The heuristics are also useful to analyze the overall usability of an application according to ergonomic criteria.In our previous studies [28], [29] we found out that most of the usability problems are related to seven heuristics: minimal actions (24.12%), flexibility and efficiency of use (13.53%), task compatibility (12.35%), help and documentation (11.18%), visibility of system status (10%), legibility (9.41%), and grouping / distinction (5.88%).If we analyze only important usability problems (major and moderate) most of them (over 80%) are related to only four heuristics.

Refining and Extending the Heuristics
Our set of usability heuristics has a general coverage as regarding the application type and technology used.In our opinion the set is large enough and specific aspects related to various applications / technologies could be addressed with specialized guidelines.In this respect, in [27] an example is given on using the software assistant and heuristics for an expert based usability evaluation of an augmented reality educational application.However, our experience with municipal websites suggested that some changes are needed.We faced several difficulties when trying to explain and document the usability problems related to user guidance and support in performing tasks that follow an administrative procedure and require filling in online documents.Although there were two heuristics related to this (task compatibility and help / documentation), none of them leads to an accurate classification of usability problems.An example of such a task is to get informed and fill in the online document needed to get a new ID card (when the old one is expired) [29].There are many situations when the ID card has to be changed.The user needs to know which documents are needed in each particular case and how to manage them.The task is simple if the instructions on how to do it and the online documents are on the same page.Unfortunately, the information architecture is varying along municipal websites and does not follow a pattern of organization.In many cases, all online documents are placed together on a separate page, which is increasing the workload and makes it difficult to find the needed document.In other cases some municipality services have a different website and either this in not clearly mentioned or no links are provided.It worth to be mentioned that heuristics 22 (task compatibility) and 23 (task guidance) are addressing different aspects.Task compatibility refers to task characteristics, such as: familiar concepts, documents, and / or common practice in the domain [1].The heuristic is more relevant for applications that presume a specific expertise of the user.Task guidance and support refers to the orientation and support provided for the user during the task completion.The heuristic is more relevant for applications that require less familiar tasks or tasks that are rarely performed.Typical examples of such tasks could be found in web reviewing systems, egovernment websites, and conference management systems.An example of such a task is requesting a new ID card from the municipality before the old one expires.The first thing a user needs is a general orientation and the procedure to follow.Ideally, it should be a step-by-step procedure, showing clearly what is to be done in each situation.Task description, required documents (original acts and / or copies), links to online documents, and the contact / support information should be all placed on the same web page.

Conclusion and Future Work
In order to answer the citizens' needs egovernment websites should provide with useful and usable content.Critical usability problems could be more effectively uncovered by taking a scenario-based approach to evaluation.Improving the usability of municipal websites requires a change in attitude and a reconsideration of the software development process.For developers, a user centered design attitude is mandatory.For evaluators, explaining and documenting usability problems is needed in order to convince developers to understand and fix them.Therefore usability heuristics represent valuable design knowledge.Heuristics are used to explain and document usability problems, to create a user centered attitude, to train novice evaluators, and to analyze the ergonomic quality of an application.When used in conjunction with usability guidelines (and we argue to do so), heuristics are useful for structuring and indexing the usability guide.Evaluation results from previous studies revealed two design flaws in municipal websites: poor information architecture and poor guidance and support for users' tasks.Two new heuristics were added in order to answer these specific usability issues.In a future work we intend to develop usability guidelines that are related to these heuristics and to extend the functionality of the software tool in order to better support the evaluation process.

Table 1 .
The revised set of usability heuristics