Keywords

1 Introduction

Nowadays we are used to interact with various mobile applications in our daily lives. Our smartphones are available almost all the time and we use them in many situations and in different places. Furthermore, we use them as an information source (e.g., mobile internet) and for orientation purposes. We use them for social inclusion, for communication and for entertainment purposes. These examples illustrate the variety of use cases for mobile applications and other digital services behind.

However, these applications and services are not accessible for all people in the same manner. Visually impaired or blind people are very often excluded, or at least limited in their use. Although offered accessibility features such as screen readers are specially designed to overcome (or at least to reduce) these barriers, their helpfulness depends on various factors, ranging from environmental conditions, over the complexity of the content in need of conversion, up to the user her or himself and her or his physical and cognitive conditions (similar to the context of use as described in [7]).

Especially the last aspect, the physical and cognitive condition of the user, limits the use of accessibility or screen reader tools very often. This becomes more obvious, when we consider the Active and Assisted Living (AAL) domain and its primary target group of older adults. Additionally to visual impairments, older adults suffer very often from other age-related physical or/and cognitive impairments. The majority of older adults represent the so-called “digital novices”, i.e., they have little experience with technology, digital services and novel interaction patterns and gestures. Designing systems, meeting the requirements of visually impaired or blind users, digital novices and users, who have potentially also other age-related diseases, is a highly challenging task.

The project Indoor and Outdoor Navigation for Increasing Independence (ION4II) addresses these challenges and aims to develop an assistive system for visually impaired or blind older adults. By using a dedicated care and residential facility as the target setting, the project focuses on the increase of the quality of assistive interaction techniques, on the development of a seamless in- and outdoor navigation service, and on the development of various personalized information exchange and security services, like personalized meal plan, personalized walking/travel times and orientation services.

This work-in-progress paper provides an overview of the developed assistive system and the user involvement within the design and development process. We will present the used methodology and the successive elaborated results, including their influence, in form of improvements to our final prototype.

2 Background and Motivation

The project was started by key partners from different research areas such as experts in localization and navigation, researchers and developers in the Human Computer Interaction (HCI) and Assistive Technology (AT) field and a professional end user organization, specialized on assisted living. For this special target group, we relied on an end user organization experienced in operating a residential and care facility for visually impaired and blind seniors (Haus Waldpension, Lower Austria). The facility offers accommodation and services for short-term vacationers and for permanent residents. Thus, Haus Waldpension provided a great setting for the User Centered Design (UCD) [1] approach applied in the development phase.

The user requirements elaboration was subdivided into two phases. The first phase involved caregivers and facility staff members. We focused on the elaboration of use cases and scenarios, which potentially help end users to become or remain informed, independent and self-confident in the dedicated residential and care facility. This phase capitalized on the empirical data, regarding wishes and residential needs, provided by experienced staff members. The second phase mainly involved primary end users. The focus was on the evaluation of their current physical conditions. The following section will discuss the elaborated results which were used as basis for the design and development of the assistive system prototype.

2.1 Use Cases and Scenarios

Use cases and scenarios are based on a list of well-known problems, provided by caregivers and facility staff members. It was reduced to the following five use cases for the assistive system:

  • Welcome tour: A guided tour through the facilities essential areas, including explanations; Targeted at new residents.

  • Indoor navigation, including explanations of near-by areas.

  • Information about the weekly meal plan, and reminding of the users’ choice.

  • Personalized appointments and notifications based on walking time estimations.

  • Guided outdoor navigation through the adjacent park including a seamless indoor-outdoor transition.

The list of use cases emphasizes that the solution addresses two main areas, namely the increase of autonomy and the management of time and place related information.

2.2 Users’ Physical Conditions

The evaluation of users’ physical condition helped us to identify supplementary and complimentary input/output modalities, and to evaluate their usefulness for the target group. The evaluation followed the human senses: (a) hearing, (b) the tactile perception and (c) the visual perception. Figure 1 illustrates the applied test setting: a hearing test, a tactile perception test and a visual perception test (from left to right).

Fig. 1.
figure 1

Test setting for the evaluation of users’ physical condition. Hearing test, tactile perception test and visual perception test (from left to right).

Full size image

Nine participants (N = 9; 5 female/4 male) with an age range from 45 to 92 and a mean age of 80.78 (15.74 SD) years were involved in the tests. Five participants (n = 5) were blind (visual function not measureable) and four participants (n = 4) were severely visual impaired according to the International Classification of Diseases ICD-10 [2]. Table 1 summarizes the evaluated results.

Table 1. Physical conditions evaluation results of nine participants
Full size table

Summary and Conclusions

  • Hearing: As already expected, users of this target group suffer from age-related hearing loss. The vocal output modality is feasible but the volume of the output should be adjustable up to 85 dB. Users preferred a female voice with an average speed of 136 words per minute.

  • Tactile Perception: Users are able to distinguish different shapes using their touch sense with a minimum high difference of 2 mm. None of the participants was capable to use the Braille lettering. Thus, an automatic conversion of visual letters into corresponding Braille letters is not applicable.

  • Visual perception: Five of nine users had no measureable visual function. The remaining four users were classified as severely visually impaired. Thus, a purely visual modality, such as a conventional Graphical User Interface (GUI) is not applicable. However, a simplified, high contrasted GUI may still be useable for some users.

3 Architecture Design

The architecture design was guided by two main aspects: Firstly, by the main application themes (increase of autonomy and information exchange) and secondly, by the target group’s accessibility challenges. The findings of the user involvement were used as a basis for the technical specification, incorporating the following requirements:

  1. (a)

    Mobile solution, with smart phone like form factor

  2. (b)

    Support of real-time indoor and outdoor localization and navigation

  3. (c)

    Support for remote manageable and personalized data (e.g., personal schedule)

  4. (d)

    Support of multimodality and adaptivity according users’ needs and wishes

Figure 2 sketches the architecture of the assistive system. It shows three main parts, namely (a) the cloud-based repository for personalized data, (b) the smartphone client for the localization, navigation and multimodal user interaction at the center, and finally (c) the localization hardware deployed in the care facility.

Fig. 2.
figure 2

Architecture of the assistive system

Full size image

Cloud-Based Services.

One of the requirements of the assistive system is the support of remotely manageable data. Most of the user’s appointments and the personal meal plan are managed by facility staff members. Thus, an easy to use and centrally manageable information and scheduling system was the appropriate solution. We decided to use existing web-based services such as ownCloud [4] and Google Calendar [5] for this purpose. These systems provide an appropriate user (data) management, which on the one hand can be accessed remotely and on the other hand integrated easily.

Smartphone Client.

The smartphone client represents the core module of the assistive system solution. It orchestrates the cloud-based services and the localization and navigation functionality. The smartphone client is also handling the personalized user interaction. This dictates that it holds an up-to-date user model and the functionality for an automatic User Interface (UI) adaptation.

One of the main goals was the design of a software solution that is easy to use and accessible for the target group. The solution should be useful for visual impaired and blind people, but nevertheless usable for people with normal vision. Thus, the multimodality and modality conversion, e.g., visual textual to the corresponding audio representation, was of outermost importance. In order to reach this goal, we decided to utilize the Multimodal Interaction Activity (MMI) Architecture. The architecture was first published as a Recommendation of the W3C Multimodal Interaction Working Group. “The MMI Architecture provides a loosely coupled architecture for multimodal user interfaces, which allows for co-resident and distributed implementations, and focuses on the use of well-defined interfaces between its constituents [3].”

Localization Hardware.

The project has high requirements regarding the accuracy of the user localization. It is well known, that conventional solutions, like the built-in position detection of smartphones or tablet devices, provide unreliable results. Considering the special target group, it becomes obvious, that the aimed assistive system requires additional hardware in order to provide an accurate user position. We used state-of-the-art indoor position technology and equipped the entire facility with small, iBeacon-compatible [6] devices. These devices use the Bluetooth technology to communicate with mobile devices in order to determine their approximate location.

4 User Experience Evaluation

The iterative design and evaluation process of the UCD approach helped us to build a stable and accessible prototype. In total, we conducted six user involvement and User Experience (UX) evaluation actions with partially different stakeholders. Table 2 summarizes the evaluation process including the involved participants and the main objectives of the investigation.

Table 2. Summary of the user experience evaluation process during the project
Full size table

4.1 Mockup Tests

The user experience evaluation started with a mockup test in September 2015. For this purpose, we developed a functional, interactive, smartphone-based mockup, following the recommendations of the user requirements phase (see Sect. 2). Figure 3 depicts two screenshots of the GUI that were presented to the end users on two mobile devices (Samsung Galaxy Tab4 [9] and Samsung Galaxy S4 [10]). Next to the graphical representation and touchable input we also used the audio output modality. The audio channel was rendered on the built-in speakers of the used mobile devices and on a bone conduction headphone (Aftershokz, Bluez 2 [8]). This early mockup neither provided actual localization nor navigation functionality. The mockup was controlled by an additional device, to pre-test the navigation features, by manually triggering concrete navigation instructions on the user’s device remotely.

Fig. 3.
figure 3

Screenshots of the GUI presented to end users during the first mockup tests.

Full size image

Five participants (N = 5; 2 female/3 male) with an age range from 46 to 90 and a mean age of 74.4 (18.64 SD) years were involved in the tests. Four participants (n = 4) were blind (visual function not measureable) and one participant (n = 1) was severely visual impaired. Only one participant (n = 1) used his smartphone on a regular basis. The remaining others (n = 4) owned a smartphone, but used it just for incoming calls.

Mockup Test Evaluation Results.

As expected, none of the participants was able to read the presented text nor to clearly to identify any of the presented pictograms. Only the navigation pictogram was partly recognized by the participant with measureable visual function. The audio output modality was only conditionally suitable for the participants. The implemented functionality (read aloud by touch) was not practicable because participant experienced difficulties to orient themselves on the multi-line GUI. Even the participant familiar with smartphone screen readers was unable to operate the mockup. The person had troubles to estimate and identify the location of the pictograms and of the interactive elements.

Implications for Further Developments

  • Plain GUI-based interaction methods are not applicable for the target group. The prototype may support a GUI, but it must also be operable without clear knowledge of pictogram positions.

  • Conventional screen reader functionality, reading aloud the textual content of GUI elements, is not applicable for the target group. Although audio output represents the main output modality for the majority, a plain GUI-to-audio conversion is insufficient and does not provide the required accessibility.

4.2 Expert Tests (I, II and III)

Since our solution is targeting people with normal vision as well as visually impaired people, we considered also other users as potential experts regarding the user experience evaluation. This was especially true for users who were confronted with the prototypes unsupervised and for the first time. Thus, we involved also this target group into our evaluation and we had in total three phases of expert tests. All tests were conducted at the real setting at Haus Waldpension and under real conditions (excluding potential long-term aspects).

Figure 4 illustrates the test situations during the expert test I (left side) and the expert test II (right side). The focus was mainly on the evaluation of the indoor position and navigation accuracy, but the tests contributed also towards usability improvements. Users were asked to follow the presented instructions and to interact with the system. In doing so, we evaluated aspects like instruction timing and latency, possibilities to repeat instructions, volume, pitch and voice of the audio output and the sentence style and syntax regarding the understandability of the resulting Text To Speech (TTS) output.

Fig. 4.
figure 4

Test situation for expert test I (left side) and the expert test II (right side).

Full size image

Five experts (N = 5) were involved in the tests, the group was composed of two UI and architecture developers (n = 2), two indoor position experts (n = 2) and one navigation and routing expert (n = 1). All experts were members of the ION4II consortium. Expert test I and II were run as one-day tests whereas the expert test III was run as a workshop-like improvement and evaluation test lasting for three days.

Expert Tests Evaluation Results.

Although the focus of the expert tests was mainly on the evaluation of the accuracy of the indoor localization and navigation, we observed additionally several usability issues. For instance, it turned out that compass-based direction instructions (e.g., turn south-west) and distance indications in meters were hard to interpret. Also, there is a need for continuous position updates and repeated output of the current position in order to confirm that the user is still following the right path. Additionally, we had also several timing problems causing a delay of navigation instructions.

Implications for Further Developments

  • Navigation instructions should support distance estimations in step-units rather than in meters. Also a compass-card-based direction instruction should be replaced by simple directions like left, right, straight ahead etc.

  • Implementation of periodic audio output functionality and the possibility to repeat the last instruction via one click.

4.3 Friendly User Test

Results gained from the expert tests helped us to improve and to optimize the prototype towards general usability. However, to enhance the accessibility for the primary target group, we relied on concrete interaction experiences from visually impaired people. For this reason we invited two (N = 2) so-called “friendly users”, to evaluate our early prototype in the field. The visually impaired participants where outside the AAL target group (both participants were under 50 years old and were technologically skilled). Both participants tested the menu guidance, the calendar and the indoor navigation feature in Haus Waldpension.

Figure 5 Screenshots of the prototypes visualizing three direct clickable elements (left) and one direct clickable element and two menu navigation areas (right) visualizes the prototypes used during the friendly user tests. The left image illustrates a version with three direct clickable interactive elements and the right image illustrates a version with one clickable interactive element (middle area) and two menu navigation areas (top and bottom area of the threefold image).

Fig. 5.
figure 5

Screenshots of the prototypes visualizing three direct clickable elements (left) and one direct clickable element and two menu navigation areas (right)

Full size image

Friendly User Test Evaluation Results.

We identified several shortcomings of the indoor navigation use case. The problems ranged from instruction language mix (e.g., “turn sharp-left” instruction read aloud by the German TTS), over distance estimations with three digits after the decimal point, to contradictory navigation instructions. We identified several problems from the usability perspective.

In contrast to blind participants, people with minimal visual perception would appreciate graphical representation of interactive elements, e.g., colored areas for menu navigation areas. Also, there is a need for audio-based self-explanations of interactive elements triggered e.g., by the first touch. Next, friendly users emphasized that every change on the GUI had to be explained by the audio output modality precisely. As an example, the visualization of an empty set of user appointments requires a corresponding audio representation, e.g., in form of an audio output “you do not have any appointments scheduled for today”.

Implications for Further Developments.

The friendly user tests emphasized that there was still a need for a visual feedback for people with minimal visual perception.

However, the “friendly user” test was performed before the last Expert Test III (see also Table 2). At this point we were confident to eliminate all indoor navigation problems before the start of the short-term trials. Unfortunately, the following Expert Test III clarified that the aimed indoor navigation could not reach the necessary stability within the remained project period. In order to maximize the project outcome for the end users we decided to offer an indoor orientation service instead of a full turn-by-turn indoor navigation service.

The improvements on the indoor position localization (accuracy ≤ 2 m) facilitated a precise real-time estimation of the users’ positions. Thus, we annotated navigation zones with additional semantic information that could be directly delivered to the end user. Doing so, we enabled users to orient themselves within the facility by querying their current position and the nearby point of interests.

4.4 Short-Term Field Trials

Short-term field trials were performed in November 2016 with the main scope on the evaluation of the overall user experience of the pre-final prototype. Figure 6 illustrates two screenshots of the prototype presented to the end users. The left image illustrates a version with three interactive elements (menu backward on the top, menu item in the middle and menu forward on the bottom) and the right image illustrates a version with two interactive elements (menu item on the top and menu forward on the bottom). Besides these differences, both versions worked in the same manner. A single click on any interactive area causes the TTS output, explaining the current selected menu item in the middle. The prototype provided also two different menu item activation styles. In one version, the selected item was activated by a two taps and in the other version by a single tap followed by holding for one second.

Fig. 6.
figure 6

Screenshots of the GUI presented to end users during the short-term field trials.

Full size image

Twelve participants (N = 12; 5 female/7 male) with an age range from 47 to 88 and a mean age of 68.67 (12.11 SD) years were involved in the tests. Three participants (n = 3) were blind (visual function not measureable) and nine participants (n = 9) were severe visually impaired. Three participants (n = 3) did not use any digital devices. Seven participants (n = 7) used a smartphone on regular basis and the remaining two participants (n = 2) used the PC at least once a week.

Short-Term Field Trials Evaluation Results.

One third of the participants could independently use the application within a few seconds. The second third required one or two explanations and the last third had comprehension problems even after few explanation rounds. It turned out that participants with minimal visual perceptions were able to use the application very well, whereas blind participants required tactile feedback in order to be able to identify interactive elements on the smartphone screen. Regarding the two different interaction methods, participants preferred the version with two interactive areas. This was mainly because of the reduction of elements and the increase of touchable areas for the remaining two interaction elements. Regarding the activation style and audio output, participants preferred two taps for the activation of the selected menu item. The TTS output was clearly understandable for all participants.

5 Conclusions and Outlook

The short-term field trials highlighted that at least two thirds of the participants were able to interact with the application independently. However, in order to make the application accessible to all users, the final prototype requires additional tactile feedback that supports users to distinguish between different interactive areas. We will implement a haptic barrier on the phone screen in the final prototype and by doing so, we hope to provide additional support, especially for blind users.

We are planning to invite at least fifteen (N = 15) participants into the long-term field trials and to test the prototype over a period of 8 weeks. We will conduct a pre-interview in the beginning of the trial, two interviews during the trial (scheduled for the week 2 and the week 5) and a final workshop including all participants at the end of the trial phase. The pre-interview aims to inquire users’ expectations regarding the assistive system. The following two interviews aim to inquire users’ experiences during the trial and to support users in case of any obscurities. And the final workshop aims to recapitulate experiences, to work out positive and negative aspects and to generate ideas for further developments and application possibilities.