An Exploratory Study to Assess Digital Map Zoom/Pan/Rotate Methods with HoloLens

: Geographical map display plays an important part of a GIS (Geographical Information System). The usability of a map display is certainly depends on how easily user navigates through spatial data and selects features on it. Currently, desktop computer based GIS applications uses mouse movements, buttons and scroll for a set of functions such as zoom, pan and rotate. However currently Hololens supports only gaze, air tap gesture and voice commands as an input. Although the functionality looks simple, it is challenging to ﬁnd optimal solution by using input devices for those functions even while creating in a desktop application. This study aims to assess an optimal way to enable these functions on hologram maps by investigating its validity and usability.


Introduction
Today, zoom and pan functions for 2D geographical map display are standardized for software and applications whether they are commercial or open source.Google Maps, Bing Map, ArcGIS, Yandex, OpenStreetMap, MapBox use same mouse interactions and performs same behavior for zoom and pan functionality.Mouse drag is used for pan, scroll is used for zoom in and out without changing mouse coordinates, double click is used for zoom in, and single click is used for selection.However, middle button and right click buttons are not standardized.Google Maps and ArcGIS do not assign any functionality for middle button and uses right click to open a context menu.Bing Map middle button drag action opens a zoom rectangle and zooms to drawn area after mouse release.MapBox rotates map by right click dragging.There are also zoom/pan graphical user interface buttons and controls placed on map as a mouse interaction alternative but they are rarely used.Popular 3D map viewers such as Google Earth and Nasa World Wind use the same mouse buttons as 2D viewers and add functionality to middle or right click dragging actions to change map rotation and elevation.The usage of zoom and pan functions are not standardized in 2000s era for desktop applications [1].Although several methods proposed for zoom and pan [2], they are currently unified on 2D desktop geographical map viewers.Zoom and pan evolution in desktop computer are spread over lots of years [3].Pinch gesture for multitouch devices has played important role in IPhone/IPad success [4].After IPhone success multitouch devices have gained popularity and pinch gesture is used for zoom/rotate, single touch movement is used for pan and single touch is used for selection mostly in mobile geographical maps.Hololens is a standalone augmented reality headset, which puts holograms on real world.This brings a new state of the art hologram technology to end users and geographical map applications will be multiplied as device gain popularity.Users interact with holograms with gaze, air tap gesture and voice commands.It is obvious that somehow geographical map applications in hololens should make zoom/pan and rotate functions easily to achieve success and grow up.Effectiveness of zoom/pan gestures [5,6] on a wall displayed geographical maps by using external motion sensors as an input is not applicable for Hololens since hand gesture recognition area of Hololens is restricted.While using Hololens, hands should be in Hololens viewport for tracking since Kinect like sensors are placed on headset.Hololens user shouldn't have to be stayed on a Kinect sensor since sensor is moved with Hololens wearer.For this reason, some gestures which are offered for zoom/pan for very large displays are also not applicable for Hololens [7].Operator only changed its position to view objects in a more detail which is not possible with 2D displays.
Although this natural zoom/pan is possible, zoom/pan operations with gestures should also be performed in Hololens geographical maps since geographical area of map may be too large and operator may not want to move.Kinect, glove/touch like sensor or external camera usage may have advantages over methods presented in this work but usage of devices other than Hololens is out of scope.Evolution and principles of zoom/pan for geographical map applications with single and multi-pointer inputs are presented in next chapter, after that a gesture for zoom/pan/rotate offered for Hololens is explained, usability analysis performed by experts shall be found before results and discussion section.
The contributions of this work can be described as follows: First, our goal is to explain principles and evolution of zoom/pan operations in geographical map applications.Secondly, we investigate methods and offers a brand new gesture for zoom/pan operations for geographical map applications in Hololens.Lastly, we conduct a set of expert review sessions to provide opinions about Hololens geographical map applications, zoom/pan operations and offered gesture by this work.

Principles of Zoom/Pan/Rotate Operations with Pointers
Getting closer to natural behavior of humans to investigate real world is the key for gesture success on synthetic environments [9].Universally accepted geographical map panning with single pointer is based on dragging a paper map on a table and mimics natural movement.For this reason pan mouse mode and cursor of pointer is iconized as hand icon in GIS applications.Pan operation using single pointing device can be best described by dragging a 2D geographical map with a pointer.Pointer may be a mouse or a single touch screen stimulation.Suppose that P1 is the pixel coordinate of a pointer location and the geographical coordinate of P1 in the pointer press time is Istanbul.When user drags pointer in the pan mode than pixel coordinate of the pointer is changed (P2) while geographical coordinate of the pointer location (Istanbul) doesn't as shown in Figure 1.In the pan mode map scale doesn't changed but viewport of display is arranged to support dragging pointer coordinate is transformed to same geographical coordinate when dragging started.So if there is a transformation function T(P, time) which transforms display coordinates to geographical coordinate during pan operation then; transformation of pointing device screen coordinate to geographical coordinate should be same during drag operation in order to achieve successful pan.
Investigation of a paper map on a table with two hands focuses user to area of interest by dragging and rotation natural movements.Based on this natural behavior, when there are two pointing devices, which is the case for multitouch pinch gesture, map center, scale and rotation is changed to focus user to area of interest.When user press P1 and P2 with two fingers and moves fingers to point P3 and P4 then geographical transformation of P1 at start time and P3 at end time will be same.Geographical positions are also same for P2 at start time and P4 at end time as shown in Figure 2. Map software which implements rendering makes necessary transformation in order to accomplish pressed positions stability in geographical coordinates.When the distance between two points is increased then map zooms in.This is the similar natural behavior of getting closer to map by using hands or eyes.When the rotation between two fingers is changed than map is rotated and this is also a natural way of rotating a paper map.However geographical coordinates of touch points remains same during those operations like single pointing device pan.This principle in map display software gives Hololens can recognize voice commands."Panleft, pan right, pan up, pan down, zoom in map, zoom out map" like voice commands may be used in order to zoom/pan functions.Map application can show hints to user for applicable voice commands to help ease of use.However there is no geographical map application that uses voice to navigate currently.Figure 4 shows natural movements by using two hand to focus on area of interest or expand an object.Two Hand Pinch (THP) is offered as the most convenient way of doing zoom/pan functions on Hololens geographical map applications and it is based on this natural movement and multitouch pinch.The gesture is briefly mentioned with a few sentences in another work [10].Similar gesture also proposed for uniform scaling of 3D objects in a virtual reality environment [11].The presented usage of THP gesture in a Hololens environment for zoom/pan operations of geographical maps is supposed to be unique.Air tap of one hand is supposed to be equivalent of touching a surface on a multitocuh screen.Moving of one hand after air tap is the equivalent of multitouch drag.A ray, which is defined by eye position and air tap position, is used to find intersection point on scene.P1 is the first air tap point and P2 is second point on scene.After two air tap event occurred, operation is started and new points on scene are found by intersecting camera and hand positions.They supposed to be P3 and P4.The critical point is that, in order to perform scale, pan and rotation operations correctly geographical coordinates of P1 and P3, P2 and P4 should be same.Geographical map software on Hololens should make scale, rotation and transformation operations on scene in order to achieve this.

Hololens Geographical Map Applications Usability Anaylsis
To measure the user experience for the proposed Hololens application, we first defined the measurable components of user experience.Consequently, we conducted a systematic process to analyze the usability.Firstly, five usability experts selected for the assessment.Next, familiarization phase was performed in which zoom/pan principles, gestures, THP were introduced and ultimately experts familiarized with Hololens and its capabilities.Thirdly, experts experienced HoloMap [12] and HoloFlight [13] and the prototype application.Finally, a usability assessment questionnaire and a set of interviews were conducted.
Figure 6 shows the holistic view of the research process.

Selection of Experts Familiarization Phase
Experience Phase Interview Phase Figure 6.The systematic research process of the study A interview with 5 experts were conducted to investigate geographical map applications usability and pan/zoom/rotate usage in Hololens.First zoom/pan principles in pointer devices, THP gesture and Hololens gestures with limitations were introduced to experts.After that three Hololens application were used by experts about half an hour by considering how users can effectively investigate and use geographical data.First application is HoloMap [12] application which shows a city with 3D buildings and supports zoom/in zoom/out buttons and air tap pan.Seconds application is HoloFlight [13] and it is an air traffic control application prototype which shows offline prerecorded data of airplanes on a 3D geographical map.The last application is a command and control application prototype developed by ANONYMOUS which renders a 3D geographical map and moving 3D airplane/ship models on a geographical map 1 .After this experience, some short questions related to geographical map applications usability and zoom/pan methods in Hololens are asked to experts with three possible answers which are positive, neutral and negative.Expert  1.
All experts agree with geographical map applications grow in popularity as Hololens becomes more accessible.All experts agree that pan using air tap drag is convenient way for panning geographical map.All experts clearly understand THP usage and prefer THP gesture usage in Hololens geographical map applications for zoom in/out although they are not experience working THP supporting Hololens application and they can not prefer/offer another gesture than THP by considering Hololens limitations.In all cases, the experts agree with voice command usage for zoom/pan may be useful.The majority of participants agree that Hololens geographical map applications are not mature and while one is neutral and they all agree that current applications will be more user friendly if they support THP.Three experts think that movement around scene to investigate geographical map is useful while one is negative and one is neutral.They find buttons for zoom and pan may be useful while two expert didn't.A common view amongst experts is that external device such as touch pad for zoom/pan may be useful while one expert is negative and one other is neutral.
We conduct a usability rating and evaluation assessment with five experts rated eleven usability cases.To observe how much participants agree with each other, we performed an "interrater reliability" (IRR) calculation [8].The goal was to assess the level of agreement among the experts (i.e.usability raters) which was expected in a range between 0.5 to 0.9 at statistically significant levels (p< 0.05).The results were combined where raters rated each step of a given task on a simple 1-to-3 scale that can be selected as negative, neutral or positive as shown in the Table 1.The Krippendorff's alpha was found 0.646 (based on five experts among 11 cases with 55 decisions), which indicated that there was a level of agreement among the experts on the use cases.
After conducting the analysis, we interviewed experts about their opinions regarding their experience with Hololens and geographical map applications: As one expert said: "Air tap is hard to learn at first.Movement around scene is more useful for zoom/and pan than other methods.If buttons are used for zoom and pan than they should be big." As one interviewee put it: "It is hard and tiring to focus by gaze.Focus point is not stable and there is a precision problem.For this reason if voice commands are used for zoom then a zooming point should be fixed before command.When I air tap it obscures scene.I have a difficulty to get used to air tap.It will be good to place scene in a comfortable place.It is appropriate to use Hololens in geographical map applications which doesn't need to much interaction" One expert reported that: "I don't prefer voice commands instead of THP but voice command may be usefull

Figure 1 .
Figure 1.Holographic view of a geographical map application in Hololens

Figure 2 .
Figure 2. Pan operation using a pointer on a 2D display.

Figure 3 .
Figure 3. Pinch gesture for geographical map

Figure 4 .
Figure 4. Natural two hand movement examples

1 andFigure 5 .Figure 7 .
Figure 5. THP movement Kılınç, M. Yilmaz / An Exploratory Study to Assess Digital Map Zoom/Pan/Rotate Methods with HoloLens user a feeling of firmly hold of a paper map during map investigation (see Figure3).Zoom/pan/rotate operations with multitouch have superiority to single pointer operations since it is widely used and navigation to interested area is much faster.For this reason, application of multitouch pinch like gesture to Hololens is valuable.
İ.2.1.Two hand pinch (THP) with hololensCurrent Hololens geographical map applications implement zoom functions with GUI buttons.User gaze to zoom buttons and makes air tap in order to zoom in and zoom out.Pan operation is accomplished like single pointing device pan operation with HoloMap application.Other geographical map applications doesn't support pan operations yet.Currently no geographical map application support map rotation.However, Hololens user can rotate around scene in order to view scene in different angles.