Keywords

1 Introduction to the Technologies

Three display technology efforts are described that shared HF-related “growing pains” during development. These technologies benefit distinct aviation, medical, or transportation end-user communities, but common HF principles apply to their preparation for transition. We start with an introduction to the three technology end-users:

  1. 1.

    The first example we discuss derives from the military aviation setting. When a new display is developed to decrease the incidence of SD episodes, most of the focus of the scientists, engineers, and software developers is placed upon whether an aviator will be able to readily learn to use the new display and will find it helpful.

  2. 2.

    A second example derives from the health care setting: when a display is developed to assist balance patients with their rehabilitation, emphasis must be placed upon whether the patients can rapidly and intuitively use the enhanced sway cues.

  3. 3.

    The third example applies (but is not limited) to automobile traffic. When a collision avoidance display is developed to warn automobile operators, pedestrians, or police officers (making a traffic stop) concerning approaching vehicles, careful thought must be devoted to whether end-users can easily employ the display.

These are important considerations that have instigated HF development practices to ensure the satisfaction of such end-users, including subject-matter-expert evaluations, job/task analyses, and usability evaluations. Nevertheless, in order to be widely accepted and disseminated, a new display must be readily usable by many persons in addition to the final user, including members of the original development team (e.g., the engineer or programmer), display demonstration spokespersons, and funding sponsors or decision-leaders receiving demonstrations. If any of the people in this human chain leading to the end-user cannot understand the product easily, technology development may break down and the product may not reach fruition. Therefore, we recommend technology developers broaden the saying “know thy user” beyond a consideration of the end-user. While the end-user’s satisfaction with the final product is necessary, there are several reasons why it is not sufficient and designing for usability must extend to other people dealing with product prototypes and demonstrations:

  1. 1.

    The end-user’s knowledge, skills, and abilities may differ from the other persons in the chain of technology development. For example, in the military aviation setting, it would be inappropriate to demonstrate to a funding sponsor (who may lack advanced knowledge or currency concerning the piloting of aircraft) the full set of product features one may show to a rated instructor pilot. Conversely, pilots may be less receptive to a “lite” system, which rapidly demonstrates a “proof of concept” to a decision-maker but lacks the features and feel pilots expect. In fact, several of the authors have demonstrated the Tactile Situation Awareness System (TSAS) [2] to pilots who initially resisted the idea when they experienced the TSAS in a primitive part-task flight simulator, only to endorse it once experienced in a fully-featured Army flight simulator or helicopter. Unfortunately, the converse problem arises when demonstrating the display to key decision-makers, i.e., it is possible to design a sophisticated flight system that interests pilots while not convincing a decision leader to recommend the system for transition to the cockpit. Clearly, the development team must balance the needs of these different users and cannot assume that a satisfied end-user ensures acceptance by other people in the human chain of technology development and transition.

  2. 2.

    Conversely, even when the end-users do have reduced needs or knowledge relative to others in the chain of product development, the technology will never be adopted for use solely by considering end-users’ needs. For example, in the health care setting, the visible software features and demonstration modes for a device meant to aid balance and prevent falls must be different for physical therapists, patients, or hospital administrators. There are two main system “users” whose needs differ: the patient and the patient’s physical therapist. Any system that is too time-consuming for a physical therapist to learn or use will end up collecting dust, even if the system may have benefitted the patient had it been designed appropriately. Also, a physical therapist or clinician needs to access advanced features that would exceed the understanding of a patient. It is common practice to create alternative interfaces for users with differing expertise or roles [1], e.g., “wizards” for beginners and expert modes for advanced users. This is one approach to designing for multiple users that can be of benefit to the aviation, health care, and transportation display development efforts discussed in this report.

  3. 3.

    Even in the seemingly simple case of a display system to warn of impending vehicle collision (where an expert is not required to grasp the problem), usability will mean different things to different people within the original development team. For example, the best collision warning display will not be adopted by end-users or customers if its features cannot be readily explained and reliably demonstrated without error by key demonstrators (e.g., the team scientists or leaders who often serve also as spokespersons for the product during presentations where technical support personnel may not be onsite). A new display may save lives that would have been lost due to vehicle collisions, but it will garner no interest if it takes longer to prepare or demonstrate than the amount of time the team was permitted by a key decision leader (e.g., 5 min to set-up and 5 min to demonstrate). In any of the three technology cases studied in this report, the systems must be optimized for satisfactory use by all potential users.Footnote 1

2 Case Study 1: HF Challenges During the Development of Display Technology to Improve Aviation Safety

The first case study evaluates a demonstration system for a TSAS for pilots [2, 3], wherein users operate a simulated helicopter while receiving vibratory navigation cues in order to understand how a touch cue can aid with SA. The vibrotactile display is integrated with a semi-portable, part-task flight simulator, but it is driven by its own native hardware, which was developed independently. The original version of the simulation was fairly user-friendly and satisfactorily integrated with the hardware. The system required an upgrade to make the graphics modern, add more demonstrations, and permit the user and demonstrator to sit side-by-side in the cockpit and easily transfer control to one another. Unfortunately, the upgrade introduced usability issues due to a failure to properly consider several HF principles of display design. First, the interface lacked flexibility and efficiency of use [4], as evidenced by the fact that it was more difficult to demonstrate the same scenarios as had been shown in the past. When sponsors or decision-leaders see a demonstrator restart a simulation or fruitlessly search through menus, this may cause them to conclude that the underlying display concept is unreliable or unrefined. Such was the case with the new TSAS flight simulator. The demonstrator had to compensate for numerous software idiosyncrasies that increased setup time and the likelihood of human error. Indeed, demonstrators had to receive significant training to effectively cope with poor system design and prevent failed demonstrations to influential persons. This runs contrary to Mayhew’s [5] admonition that ease-of-use should be emphasized when designing systems with high frequency of mandatory usage. This sentiment is echoed by Shneiderman [1], who emphasizes the importance of ease-of-use, particularly for non-technical “users” (adopting our wide usage of the word being advocated in this report). If key technical personnel are not available for demonstrations, other team spokespersons must serve as demonstrators, making ease-of-use essential.

The simulation also failed to design for control stability, an HF principle of control design [6]. To initiate motion of the simulated helicopter, the user was required to maximally displace the joystick/cyclic. This change was implemented to simplify demonstrations, but it introduced an inordinately large dead spaceFootnote 2 in the joystick, creating an inappropriately low gain (i.e., ratio between change in outputs as a function of change in inputs, [6]), which forced the user to “max out” the controls to resume even slow forward flight. Moderate gain levels are usually considered more desirable for balancing responsiveness and stability [7].

Another problem with the upgraded simulator is that once motion had been achieved, if the joystick was moved near the center/neutral position (even momentarily), the simulated aircraft motion stopped. This was not desirable because the TSAS only activates when one is moving and the point of the demonstration is to feel and understand the functioning of the TSAS display.

In combination, the need to move the joystick all the way over to move the aircraft and the cessation of aircraft motion when the joystick was near center left the end-user feeling as if he/she must “fight” the controls. This reaction was consistent with past research suggesting increased dead space decreases user control accuracy during tracking tasks [8]. Such problems could decrease the interest of a decision-leader from whom one is seeking endorsement of the underlying display concept being conveyed by the prototype. Even when a technical member of the original development team learned to compensate for this problem rapidly during situations where the “demonstratee” was merely in the role of a passenger, the problem was still obvious to the demonstratee, due to the repeated starting and stopping of the simulated motion.

Control instability also contributed to the time-consuming and error-prone nature of the altitude control lock. Altitude control is a feature of flight simulators that allows the current altitude to be locked into place. This is helpful to demonstrators (spokespersons or members of the development team) when presenting the TSAS, e.g., it alleviates the need for a spokesperson to focus on altitude control while explaining other capabilities of the TSAS. In the previous version of TSAS, when altitude lock was deactivated, approximately 30 % of the simulated helicopter’s available lifting power was required when manipulating the collective. Unfortunately, deactivating altitude lock in the upgraded version of the simulator initiated a rapid freefall that could only be counteracted with approximately 90 % of the simulated lifting power, requiring extreme excursion of the collective. Crash recovery thus became more difficult, which led to frequent system resets and unimpressed observers.

In the situation described above, the new system neglected design principles such as flexibility, efficiency of use, and error tolerance [1]. The control scheme was unforgiving to novices, yet also lacking in realism experienced pilots appreciate. A better design strategy is to implement realistic controls and flight dynamics for experts, but have the option for an easier “arcade mode” for demonstration to non-pilots. The users also should be able to adjust the collective easily. This can be done by modifying flight control to produce a more responsive system with moderate gain [7].

A problem presented by the updated software occurred during a tank-hunting scenario. TSAS can guide the pilot to target locations (via tactile cues on the body corresponding to the direction the pilot should head). Initially, the first target (a tank) was located in a sandy area. When the helicopter flew over this area, a dust storm was created to demonstrate the usefulness of TSAS upon inadvertent entry into a degraded visual environment (DVE). Unfortunately, the dust storm overloaded the computer’s processor, causing system lag. Time lags greatly degrad a user’s ability to control a system and increase user errors [6]. Another related issue was that the “sand pit” from which the dust storm originated was surrounded by green grass, trees, lakes, and small, grass-covered rolling hills. Therefore, this aspect of the system failed to be consistent with the user’s mental model [9] of likely environmental features. Design constraints should be implemented to make the simulation more natural and prevent users from performing actions that exceed system capabilities. A principle of HF design is the use of constraints, i.e., design elements that limit the ways in which users can interact with systems [10].

Another problem with the new software was the misleading display of system status, which violated design principles of visibility, feedback, and consistency [5]. The user was always shown a visual indicator of the current “mode”. The user could choose to work in “advanced mode” or “easy mode”. This was a good feature, but upon initialization, when the simulator was in “advanced mode”, it was incorrectly labeled as “easy mode”. If the user then chose to switch into “easy mode”, the display changed to “advanced mode”. A similar feedback problem arose whenever the software began to experience lag (as in the “sand pit” problem in the previous paragraph). Under normal circumstances, highlighted buttons indicated activation (e.g., when altitude control was activated, its corresponding button stayed highlighted until deactivated). Unfortunately, if a button was selected just before a system slow-down, then after the system recovered that button remained highlighted, despite the fact that the command was never executed. This lack of consistency and accurate feedback for user inputs and status displays led to frequent mode errors, wherein the users interacted with the system inappropriately for the active mode [10, 11], typically because of ignorance or memory lapses regarding which mode is presently active. In our simulation, mode errors occurred because the design induced them. A principle of display design is to provide visibility regarding system status [4]. Users should be kept informed of which mode is active. Furthermore, the principle of feedback mandates that users be provided with information regarding the effectiveness of their inputs [4, 5].

The new software violated these design principles: first, by providing unreliable information regarding the current active mode, and second, by providing feedback to users that failed to reflect actual mode changes. This placed a greater memory burden on demonstrators to remember the idiosyncrasies of the simulator. Good HF design requires reducing memory load and instead placing knowledge in the world [12].

A final problem was the placement of the crosshairs for aiming missiles (during the shooting portion of the simulation), which violated the principle of consistency and use of real world conventions [4, 5]. The new version of the simulator software relocated the crosshairs to the bottom left quadrant of the screen, which failed to match users’ expectations based upon experience with other systems, including previous versions of the TSAS. Crosshairs for simulations emulating a first-person, egocentric perspective are usually centered. Crosshairs not only represent a tool for “sighting” weapons, but also provide the primary anchor for a user to orient his or her personal perspective. By decentering the crosshairs, a perceived misalignment between a user’s ego-center and the simulation can arise, resulting in inaccurate targeting, adoption of unintuitive compensatory aiming strategies, and awkward flight patterns. In the new version, weapons fire still tracked centrally, as though the crosshairs were located in the center of the screen. These design flaws failed to conform to platform interface or real-world conventions [4]. The new system was awkward for demonstration participants to use because they could neither use it in the same manner as other fixed-reticle flight simulators, nor could they hit targets despite aiming perfectly.

3 Case Study 2: HF Challenges During the Development of Medical Technology to Aid with Balance

Our second case study involves a tactile balance assistance display [13, 14] designed to help orient and rehabilitate people with balance disorders. The technology integrated a balance platform with a wearable vibrotactile belt. The original technology was represented visually on a computer screen from an allocentric top-down perspective and was easy to demonstrate. The user’s center of pressure (represented by a dot) was surrounded by a virtual ring that contracted slowly, requiring the user to maintain balance to keep from colliding with virtual walls through postural tilt and sway. Swaying forward would cause a tactile collision with a virtual wall, intuitively triggering vibrotactile feedback from that direction. The user then compensated by moving in the opposite direction, typically to a more upright posture. By the point at which the shrinking circle had reached its smallest size, the user was required to maintain a consistent upright posture to keep the dot centered and avoid receiving tactile cues. If the user could not avoid hitting the edges of the circle, the ring would expand slightly to adapt to the user’s ability to maintain balance. The original system was well-designed and facilitated demonstration to groups of people, who could easily view its utility without wearing the tactile display.

A revised version of the balance test software was developed to integrate with the second generation of the balance-sensing and feedback hardware. Unfortunately, user-centered design for optimal demonstration was not considered as carefully. The new software eliminated the adaptive shrinking function of the circular visual feedback during the balance test and, in fact, provided a screen where no circle of stability was visible during the balance testing part of the demonstration. The user could still see his/her center-of-pressure represented as a dot moving across the screen but could not see where the limit, which triggered the tactile cue, had been set. In other words, the user had no way to know he/she had reached the static virtual circle visually, but only could do so tactually. At the end of the test, the software indicated the user’s degree of postural sway via a percentage presented via text (e.g., “You scored a 30 %”). This new design limited the potential for effective demonstrations, particularly to more than a single person at a time. Real-time visual feedback was an essential element of the original software, not only to assist the participant in learning to use the system and providing confirmation to the demonstrator that the system was operating correctly, but also to demonstrate the system to groups of observers. The new design violated the aforementioned principle of visibility by failing to provide feedback to users, irrespective of their role in the demonstration. Some people attending the demonstration may not have had time to try the system themselves, or they may have been shy or self-conscious (e.g., wondering if the tactile belt would fit around their waist), so it was important for everyone to understand what was happening to the person experiencing the tactile cues. Moreover, the test feedback provided to the user was cryptic, since users could not readily interpret the results. The display of cryptic messages violates one of Nielsen’s [4] usability design guidelines: providing a match between the system and the real world. One aspect of this guideline is to convey information through familiar conceptual models and metaphors. In the present case study, the use of a percentage can be viewed as a metaphor for a test grade, which can be consistent with users’ mental models [9], provided they also know the grading scale. A possibly superior approach is to provide test feedback using conceptual metaphors to which users can relate readily [4], such as unstable, slightly unstable, stable, or very stable.

A new mobile tablet version of the system was developed for demonstrations where a spokesperson may require a portable system that is immediately usable (e.g., for unscheduled demonstrations). This system required the user to hold the tablet against his/her chest so that the accelerometers in the tablet could detect its orientation and activate vibrations when postural sway exceeded a set threshold. The user responded by moving in the opposite direction of the vibration (e.g., vibration on the abdomen indicated to the user that he/she was leaning too far forward). The tablet also depicted the balance test visually for demonstration to observers who were not wearing the tactile display. It showed a dot, representing the user, inside two concentric circles. The dot moved in response to postural tilt, and when its movements breached the confines of the innermost circle, tactile feedback was presented in the appropriate direction and a blinking light activated on the tablet screen to convey the pulse rate. Exceeding the confines of the outermost concentric circle resulted in a faster tactile pulse rate. The size of the concentric circles was scalable via a gain control to adjust the difficulty of the balance task to each patient.

The tablet-based demonstration system enhanced portability, but its initial implementation had flaws from a human-use perspective. First, the visual feedback provided to the audience viewing the tablet screen indicated postural feedback was being provided to the front of the torso, when it was actually presented to the back. The same was true for lateral tactile cues (i.e., there was a left-right reversal). This violated the design principle of stimulus-response compatibility [15] and the principle of the moving part [16]. In this case, there was a spatial mismatch between the tactile display and visual feedback, which confused the user or people observing the demonstration. Secondly, manipulating the gain control not only changed the system’s sensitivity to postural change, but it also reversed the direction of tactile cues without providing feedback regarding this reversal. Rather than providing repellent/avoidance cues, (i.e., move away from the tactile cue), the system began to provide attractive/guidance cues. (i.e., move towards the tactile cue). The lack of visual confirmation of the mode change violated the principle of feedback [5]. Without such feedback, mode errors and confusion were commonplace, causing demonstration failures.

Many of the flaws above were corrected in the latest version of the tablet-based balance demonstrator. However, the text labels for the controls remained too small. Based upon an average viewing distance of 40 cm (as estimated in the laboratory) and an average character height of 0.013 cm, a typical lowercase letter in the tablet display subtended a visual angle of only 0.21 × 0.18 degrees (vertically and horizontally, respectively). The low-contrast display text was the equivalent of a 6-point font, violating the principle of legibility [6], and requiring demonstrators/spokespersons to memorize each button. When designing displays for dynamic environments (e.g., during locomotion or vehicular transportation) or for persons over age 65, then sans serif fonts greater than 12-point should be adopted to improve visibility [17, 18].

4 Case Study 3: HF Challenges During the Development of a Vehicle Collision Avoidance Display

Traffic collisions are discussed, but the technology also has a cueing mode to alert a pilot concerning collision with the ground.

There are a variety of circumstances where a vehicle collision warning may be useful. One circumstance is when a police officer conducts a road-side traffic stop. In this event, the officer exits his/her vehicle and approaches the stopped vehicle on foot. Often, the officer will approach the driver-side of the vehicle. This exposes the officer to risk of collision by an approaching vehicle on the roadway. Avoiding this hazard requires the officer to divide his/her attention between the motorist and the roadway, which could slow the officer’s reactions if the motorist intends to harm the officer. All states have adopted a “move-over” law, mandating that drivers must slow down and change lanes if possible when approaching a police vehicle that is pulled to the side of the road. However, police officers are still injured (some fatally) each year by approaching vehicles. Vehicle collision warning systems have yet to be leveraged to prevent these events. To customize existing technologies to accommodate this application, the optimal sensory modality and stimulus presentation must be considered. The roadside environment can be very noisy, implying that an auditory stimulus may not be optimal. A possible solution is to create a multi-modal display that includes a tactile cue to ensure a more salient stimulus. However, such a system will never be adopted if it is difficult to use or provides unhelpful or false alerts. The team designing and testing such a device should consider practical HF limitations during development to maximize the potential for utility. When developing software, utility is usually enhanced by building upon existing formats and layouts that have been proven to be efficient and intuitive, thus reducing the recurrence of poor system design. Moreover, leveraging preexisting user knowledge by retaining similarities between versions can facilitate transfer of training [19]. If reprogramming is necessary, it should incorporate HF principles such as those described in the example below.

A simple proof-of-concept tactile array was created in an effort to demonstrate a tactile cueing concept that could reduce vehicle collisions in various settings. The seat display contained a 32-tactor grid that could signal the approach of an object (via radial expansion of concentric rings of tactor activation on the surface of one’s body, e.g., the back). The formation of the prototype necessitated that a new version of the underlying tactile display software must be written to allow the system to control 32 tactors without causing the tactor controller (signal amplification and selection hardware) to fail. The new cue was well-liked by demonstration participants, but the system presented many usability problems to the demonstrator. The most outstanding software issue was one of conveyance. Ideally, instructions for using the software should be conveyed in an easily comprehensible manner, in language the user can readily comprehend [4]. This was not the case for the tactile warning system. An example of poor conveyance was the task of loading a saved file that had been created by a technical member of the development team. This task required the user to click the “menu” tab in the center of the screen, which led to another “menu” tab on the right side of the screen, which led to yet another “menu” tab to the left. This complex menu hierarchy confused users and taxed their memory unnecessarily.

A similar usability problem arose with the simple act of booting up the program for demonstration. The new version of the program required at least twice as many steps to be completed before the demonstration could begin, some of which were not intuitive. As a result, one group demonstration failed despite the fact that the spokesperson had 30 min to set-up the system and ensure the software was working. A principle of HF design is to minimize demands on long-term memory through standardization and visibility [6]. Placing regularly used functions in locations consistent with users’ mental models [9] (e.g., due to past experience), and making them quickly accessible and visible eliminates frustration and confusion caused by navigation through embedded menus. This is important because the spokesperson may not have even 30 min to get a demonstration ready for a decision-leader or group.

Another problem related to context. The user should be able to clearly and quickly understand the function of all of the software “tools”. Typically, tools are represented by icons that cue the user on what the tool does. The match between the pictorial representation and a “real world” object should be close and make use of familiar metaphors. Certain aspects of the new software, however, violated these design principles. For example, the tactile seat warning system featured a “Database” tool, which allowed the user to access the computer’s stored files. The icon for this tool was an envelope, which connoted sending or receiving files. The mismatch between the icon and associated function confused users. A better icon in this situation would be that of a vault, floppy disk, or hard drive, which would additionally conform to the principle of mental models [9] by using a common iconic representation.

A final problem encountered by users of the tactile seat warning system who wished to develop new tactor firing sequences was an absence of consistency. In one section of the software, using the keyboard shortcut “Ctrl-Z” (undo) cancelled only the previous step. In another section of the software, using the same keyboard shortcut cancelled all previous steps, violating the design principle of consistency [5] and leading to confusion and lost progress.

5 Conclusion

This report describes case studies of HF problems we have encountered during the development of several prototype display technologies, emphasizing the problems and needs specific not only to the end-user, but also to technology developers, spokespersons, decision-makers, and others in the human chain of technology development. HF practitioners will not be surprised to learn that when these problems were initially described to the programmer who had made the modifications, he felt that most of the problems simply required more training and practice on the part of the other personnel in the technology development chain. This common and understandable reaction is nevertheless a violation of the HF principle that the fault is never with the user [20]. Fortunately, this and other problems were overcome by discussion and trouble-shooting among all team members. HF professionals know that it is important to involve the end-user in the development process [20], but in all three cases studies in this report there were several people prior to the end-user who needed to be engaged in communication and pre-testing in order to make the technology successful. In Table 1, recommendations are offered for four of the people (viz., the engineer/technician, the programmer, the spokesperson, and the decision leader or sponsor) in the chain of technology development leading to the end-user. Examples are provided of problems that can arise for due to a failure to consider HF principles. The three display technologies are shown in three vertical columns, with the horizontal rows representing some of the people in the development chain.Footnote 3 A review of the table should give the reader ideas for how the challenges encountered with these applications can foster innovations in his/her own field of technology development.

Table 1. HF Recommendations for People in the Chain of Technology Development Prior to the End-User, and Problems that Arise from Not Following Recommendations (see Conclusion).
Full size table