Evaluation of Human-Computer Interface Development Tools: Problems and Promises

Abstract

The computer industry has seen an explosive emergence of user interface management system (UIMS) toolkits in the last few years. However, there are no standards for the components of such toolkits, and no procedure for systematically evaluating or comparing these toolkits. With their proliferation, ad hoc evaluations and comparisons are constantly being done, without a formal, structured approach.

This paper will describe several of the problems involved in developing an evaluation procedure for UIMS, and will report on research that is showing promise as an evaluation procedure that produces quantifiable criteria for evaluation and comparing UIMS. Such a procedure could be used, for example, for choosing a UIMS for a particular human-computer interface development environment.

The procedure we have developed generates ratings for two dimensions:

• Functionality of the UIMS being evaluated, and

• Usability of the UIMS being evaluated.

Functionality refers to what the UIMS can do; that is, what interface styles, techniques, and features it can be used to produce. Usability refers to how well the UIMS does what it can do in terms of ease of use (a subjective, qualitative rating of how easy the UIMS is to use) and human performance (an objective, quantitative rating of how efficiently the UIMS can be used to perform a task).

A significant by-product of this research is a practical taxonomy of types of human-computer interfaces, including interaction styles, features, and hardware, in addition to a taxonomy of types of interface development support provided by UIMS, and general characteristics of UIMS.