Keywords

1 Introduction

The problems of negotiating large complex public spaces in modern society such as airports, hospitals and shopping malls has resulted in the indoor navigation problem becoming the prototypical application domain for ubiquitous computing and mobile interaction researchers, with typical approaches center around the provision of expensive and novel spatial localization technologies. Navigation or wayfinding in unfamiliar indoor environments become more of a challenge for people. Dogu and Erkip [1] documented the many costs associated with having wayfinding problems such as loss of time, decreased safety, stress, and discomfort.

There are three aspects to the indoor navigation problem: the user, environment, and task. We are interested in multiple nomadic users who have no previous knowledge of the spatial environment. The indoor environment we are concerned with will typically be large, complex and crowed spaces, which may have structural similarities to functionally related environments. Experience has shown that in many situations, users struggle to find/locate appropriate navigational information using traditional signage and maps (especially when placed under time pressure), which gives rise to the need to design more effective navigational aids. Even where navigational information is provided by an environment, it is not personalized, it is not efficient (e.g. suggest generally applicable paths) and it usually places a significant cognitive load on the user.

Navigation in indoor environments is most commonly supported by the conventional methods (architectural design, signage, maps, and verbal and written instruction). However, wayfinding signs have a number of well know limitations. Dogu and Erkip [1] identify that although putting up signs is a universally acknowledged approach to prevent people from getting lost, the language or pictographs used in the signs are not always well understood. Locating wayfinding signs in itself can be difficult for people in modern commercial public spaces where there are high levels of visual noise, such as advertisements in the form of posters or digital displays. Conventional handheld paper maps, as well as stationary embedded representations of the environment (e.g. poster maps or physical models), require users to identify their locations and the locations of their destinations in order to formulate navigation plans. Furthermore, people have significant (and well documented) problems understanding spatial layout and wayfinding performance decreases with increases in floor plan complexity [2].

Although a number of mobile and pervasive computing technologies have been proposed to support indoor and outdoor navigation (i.e. location-based guidance systems) in recent years, most of solutions (e.g. [3,4,5,6,7,8]) generally seek to present maps and spatial information using multiple modalities, and in a manner sensitive to a user’s needs and spatial context. However, such systems generally suffer one or more the following shortcomings:

  • High attention and cognitive load demands;

  • Relying on expensive and unreliable sensing and tracking technologies;

  • Not supporting users undertaking multiple concurrent tasks while navigating;

  • Small user capacity;

  • Weak protection for user’s privacy.

In order to address the drawbacks above, we applied the framework of Crossmodal Displays [9, 10] to the application area of indoor navigation, that is, we explored the design of CROSSFLOW, a novel navigation system which exploited the human multimodal perception, especially crossmodal interactions [11,12,13] and utilized the synergy of both existing public displays and personal displays. Based on these features, CROSSFLOW could facilitate navigation within an unfamiliar complex indoor environment, support multiple simultaneous pedestrians accessing personalized navigational information without resorting to location tracking, and impose lower mental workload demands on users. Few navigation system found has these features and advantages. This paper mainly presents the design principles of CROSSFLOW and the design spaces of the components of CROSSFLOW so as to inspire people to come up with the better design of the system or the better solutions to the problems associated with indoor navigation.

2 CROSSFLOW Design Principles

CROSSFLOW shows multiplexed visual directional information (the public cues) publicly corresponding to all the destinations at each decision point using the public displays (e.g. projected dynamic signs), and presents the personalized vibrotactile and/or auditory cues (the private cues) via the personal displays (e.g. smartphones combining with earphones) to different users timely to indicate the particular directional information that corresponds to their destinations.

Here, visual public information and auditory/vibrotactile private cues are used because CROSSFLOW is designed for multiple simultaneous users. For this reason, the disturbance to other occupants of the public space, including those who are not using the system, should be minimized. The private cues presented through the vibrotactile or auditory (through headphones) modalities are essentially private to the users. Although there are ambiguities in the public cues, the private cues such as buzz or beeps for the user could resolve such ambiguities. The way of using auditory or tactile cues to resolve ambiguity in a visual display is closely related to the observations and experimental design of Sekuler et al. [13] and the bounce-inducing effect [13].

The implications for multimodal interface design that we inferred from the neuroscientific literature on multimodal integration and crossmodal interaction in humans [10] indicate that when the spatially incongruent multimodal cues from the spatial incongruent displays are temporally combined and presented in parallel, a user may be able to integrate the spatially incongruent but temporally synchronized multimodal information into a meaningful new percept, e.g. an intuitive directional information indication relevant to their destinations. Moreover, the perception of visual public information may be enhanced by a simultaneous auditory and/or vibrotactile private cue regardless of the spatial disparity between the cues from different modalities. Therefore, instead of having to comprehend the configuration of a public space (as when using a map), a visitor to a public space can enter his destination on his personal device, and in response, his device receives a time slot schedule (cue mapping) which defines the time slot when the visitor’s directional information corresponding to his destination would be shown in the space. Whenever the directional information is “valid”, the personal device indicates this using the private cue.

3 CROSSFLOW Prototype

Figure 1 provides the schematic diagram of a prototype of CROSSFLOW indoor navigation system we developed, which embodied our conceptual design of CROSSFLOW. The system components and functions are as follows:

Fig. 1.
figure 1

Schematic diagram of the CROSSFLOW prototype

Full size image

Public Displays.

The public display used in the prototype are projections displayed by digital projectors connected to a networked PC server. Several different designs of the visual public directional information (e.g. the fish-flow pattern) can be projected on the floor of an indoor environment.

Personal Displays.

CROSSFLOW uses smartphones (including earphones) as personal displays through which the private cues such as sound and vibration can be displayed privately.

Cue Mappings.

We define that cue mapping as the spatio-temporal relation between the public cues and private cues. The design of cue mapping can be motivated by the different paradigms and techniques for the integration of information across the sensory modalities. Two types of cue mappings, a regular cue mapping and a random cue mapping, have been created.

Duration of Cue and Time Slot.

The durations of the cue and the time slot (800 ms visual directional information, 800 ms vibration and/or 150 ms beep sound) were explored based on the findings of [14,15,16].

Public Directional Information (Public Cues).

More than four visual design alternatives of directional information were designed including the rotating arrows shown at all decision points, highlighted markers showing all possible routes, and abstract ambient patterns covering the complete floor of the indoor environment (see Fig. 2).

Fig. 2.
figure 2

The design of public cues: pointers (upper left), floor lights (upper right), ambient patterns (lower two)

Full size image

Private Cues.

A private cue should induce/invoke a subtle switch of a CROSSFLOW user’s attention to the corresponding public directional information that is (concurrently) displayed. The private cues in CROSSFLOW could be abstract signals or explicit information such as a vibration coordinated with the onset of a time slot; or, an audible high pitch sound (i.e. a “beep”) coordinated with the onset of a time slot. In our initial design we explored the combination of sound and vibration.

Software for Visual Public Directional Information Configuration.

CROSSFLOW includes a software tool, as a component of the central server, to support the design and configuration of the visual public directional information. The software tool enables a designer to graphically and interactively configure the directions of graphical elements of visual public directional information corresponding to different destinations (see Fig. 3) and specify the parameters that control the size, quantity, density, distribution areas (see Fig. 4) and dynamic properties of the individual graphical elements (rate of movement and visual persistence), as well as the duration and number of time slot within a cycle. The tool in turn generates a configuration file saved on the central server. For example, when configuring the fish-flow pattern using this software, a designer can steer the flow tendencies around obstacles and away from sites that are not intended to lie on the path to a destination through rotating a set of influential arrows (see Fig. 5), and can also add or scale the influential arrows to attain the desired patterns of flow (see Fig. 6).

Fig. 3.
figure 3

Influential arrows for configuring the directions of graphical elements of the visual public directional information corresponding to different destinations

Full size image
Fig. 4.
figure 4

An example of how to configure the distribution areas of the ambient pattern

Full size image
Fig. 5.
figure 5

An example of how a designer steers ambient pattern flow tendencies around the central area of the screen

Full size image
Fig. 6.
figure 6

An influential arrow (in red) added and scaled down for configuring the fish-flow pattern (Color figure online)

Full size image

Software for Private Cue (Auditory and Vibrotactile) Configuration.

CROSSFLOW also includes a software component installed on the smartphone, with which a user can configure the private cue. For example, the user can select a destination from a destination list, change the mode of private cue (vibration, sound, both or none) and the sound volume, and switch on/off private cues.

3.1 Usages

A user could be guided to his destination by simply selecting a numerical identifier of his destination in the destination list on his smartphone. The smartphone may either be pre-configured to synchronize with the CROSSFLOW central server, or it may communicate with the server via a wireless connection to receive the cue mapping and to synchronize the timing of the private cues. There is no need for the user’s personal mobile device to communicate with the central server once this synchronization has been achieved.

4 Conclusions and Future Works

We present our initial design exploration of CROSSFLOW, a navigation system that has potential to avoid numerous drawbacks associated with many previous indoor navigation approaches, systems and technologies. Based on the design implications of multimodal integration and crossmodal interaction for interface design, CROSSFLOW could facilitate navigation within an unfamiliar complex indoor environment, support multiple simultaneous pedestrians accessing personalized navigational information without resorting to location tracking, and impose lower mental workload demands on users. It is a relatively low-cost navigation system because the digital public displays and smartphones are common devices and featured in many common public spaces such as shopping malls and art galleries. In the initial user study on our first prototype, individual participant was given dual tasks to perform using either CROSSFLOW prototype or hand-held paper map (more details were reported in [17, 18]). The initial results are encouraging.

However, because of the nature of CROSSFLOW, the users need to use both public cues and private cues to navigate. Besides this system limitation, some others were discussed in [10, 17, 18]. Further limitations, such as the way in which the system deals with errors, as well as the usability and the user experience afforded by CROSSFLOW need to be investigated and evaluated through formal user studies in the future. Moreover, the advantages CROSSFLOW should also be validated via a user study in which multiple simultaneous users would navigate in an unfamiliar complex physical indoor public space.