Evaluation of Multi Layers Web-based GIS Approach in Retrieving Tourist Related Information

Geo-based information is getting greater importance among tourists. However, retrieving this information on the web depends heavily on the methods of dissemination. Therefore, this study intends to evaluate methods used in disseminating tourist related geo-based information on the web using partial match query, firstly, in default system which is a single layer approach and secondly, using multi layer web-based Geographic Information System (GIS) approaches. Shah Alam tourist related data are used as a test collection and are stored in a map server. Query keyword is tested using both default and multi layer systems and results are evaluated using experiments on sample data. Precision and recall are the performance measurement technique used. Findings show that multi layer webbased GIS provide enhanced capability in retrieving tourist related information as compared to default system. Therefore, in the future, web-based GIS development should utilize multi layers approach instead of the single layer method in disseminating geo-based information to users.


INTRODUCTION
Geo-based information is gaining greater importance among tourists as it allows meaningful experience with places (Achatschitz, 2006;Tussyadiah and Zach, 2012). Furthermore, the use of geo-based information is not limited to tourists but it is also used in everyday life. Acquisition of this information through the use Information and Communication Technology (ICT) has effect on everyone especially tourists. Publishing web-based Geographic Information System (GIS) maps is one of the methods used to encourage the public to acquire geo-based information using ICT. Development of geoportals that is more receptive towards tourists' requests has become abundant today (Dickinger et al., 2008;Sigala, 2009). However, the current online maps which run on a webbased GIS do not reflect users' needs (Khan and Adnan, 2010;Kyem and Saku, 2009;Plosker, 2006;Richmond, 2002). These maps need to be improved not only in terms of usability as discussed in Khan and Adnan (2010) but also in terms of their efficiency and effectiveness (Dickinger et al., 2008;Kyem and Saku, 2009;Pan and Fesenmaier, 2006a;Plosker, 2006).
Currently, researches are either focus on usability of web-based GIS applications (Khan and Adnan, 2010;Radwan, 2005;Voldán, 2010;You et al., 2007), visualization (Pontikakis and Twaroch, 2006), geoportal collaborations (Hao et al., 2010;Sigala, 2009), shortest path (Hochmair, 2009), map related information search (Chen et al., 1998) or Geographic Information Retrieval (GIR) technique and performance (Pu et al., 2009). Little concern is given in evaluating the performance of the current web-based GIS applications (Markowetz et al., 2004;Matic, 2006;Simão et al., 2009). In addition, the current off the shelf web-based GIS products have further encouraged users to use the applications without giving thoughts to the current system's performance (Plewe, 1997;Tsou, 2004;Tsou and Michael, 2003). The integration of GIR component in web-based GIS has given initial thoughts in evaluating the performance of the latter system (Larson, 1996;Martins et al., 2005;Purves and Jones, 2006;Voldán, 2010;Zhang et al., 2012). As a result, GIR evaluation technique can be used to evaluate the performance of web-based GIS (Clough et al., 2006;Martins et al., 2005). Therefore, this study intends to evaluate the performance of web-based GIS in disseminating tourist related geo-based information on the web. Two systems are evaluated. First is the single layer system which is provided by the current off the shelf web-based GIS developer. The other is a customized multi layers system. Both are evaluated and results are presented.

GEO-BASED ONLINE INFORMATION SEARCH
Maps are tourist's best friend (Richmond et al., 2003). They are an essential tool for tourist in any phase of vacation planning process: planning, during and post vacation. However, it is often overlooked by tourism providers. Tourist often finds that destination websites are equipped with textual and graphic composition but lack of spatial information such as location map. His geographic needs are often neglected. There are four concepts of query which are clearly dominant (Andreas and Volker, 2007). These concepts are habitation, accommodation, spare time and information. General object such as 'hotel' is more frequent in the query rather than specific brand of car dealers. Users who intent to buy or rent are the dominant users who make geographic related query and this leads to the concepts of goods and services. These geographic queries have a high value of commercial impact. Andreas and Volker (2007) further defined four features of geographic information needs. First, is the intention. It describes the relation between a user and a specific place. A user might want to know something about the place. Second, is coverage. It describes the specific coverage of an area. Third, is shape. It deals with looking for a specific document which can be in a form of point, polygon or polyline. Finally, fourth is distance. It describes the interpretations of nearness. Useful distance measures are determined by the means of transportation, existing traffic routes and user's intention. Therefore, tourist spatial needs involve two basic principles. First, is Tobler's first law of geography which says, "Everything is related to everything else, but near things are more related than distant things". (Tobler, 1970) Second, is Egenhofer and Mark (1995) principle that says "topology matters, metric refines". These two principles apply to the spatial needs of tourists and are discussed by a number of authors (Egenhofer and Mark, 1995;Fodness and Murray, 1999;Nicolau, 2008;Fesenmaier, 2002, 2006b;Richmond et al., 2003;Smith et al., 2007;Tobler, 1970).
Map represents things in space (Brown, 2001;Richmond et al., 2003;Robinson and Petchenik, 1976). History has proven that travellers described places they have visited or new world in a form of map. This will help others imagine what can be found in the places they visited (Richmond et al., 2003). Maps are tourist's best friends. It performs valuable task in any of tourist's journey. It is needed in various stages of a journey regardless of whether it is planning, during or post journey stage (Richmond et al., 2003). It also provides sense of space which is related to the concept of naive geography. With the existence of current technology map making has become much easier and this concept should be used as the basis to design intelligent Geographic Information System (GIS). Naive geography is a study of formal models of commonsense geographic world (Egenhofer and Mark, 1995).
Each stage in vacation planning requires the tourist to use map differently. During the planning stage, map is used to determine the location of accommodation available at destination. Tourist will then look for places of interest around the selected accommodation. They will also look for other services such as food and transport services (Pan and Fesenmaier, 2006a;Richmond et al., 2003). On the other hand during vacation, maps are used as reference. With a map in hand, tourist will feel confident and has some orientation of his location at every moment. Maps provide direction and validate tourist's expectation at actual location (Robinson and Petchenik, 1976). Finally, after a vacation, maps can provide memorable journey virtually in the mind of a tourist (Richmond et al., 2003). Thus, no doubt that maps and tourist are inseparable (Richmond et al., 2003). However, new invention and advances in communications have provided a new dimension in tourism sector. The Internet and the World Wide Web have given a new path of communication in tourism through online maps. The following sections describe the Internet and the World Wide Web in tourism context and the current online maps.
Online information search by tourist is discussed thoroughly by Pan and Fesenmaier (2006b) and Luo et al. (2004). According to a study carried out in the United States by Pan and Fesenmaier (2002) on online vacation planning, findings have shown that tourists follow a hierarchical structure of events in searching for information. They often have different semantic mental models as compared to the content which are offered online. However, the items that they searched for are similar even though they may take different steps. These items include information on accommodation, attraction, food and transportation services. Studies by other scholars also confirm that these items are among the types of information required by tourist starting from planning until the end of a vacation (Ekmekcioglu et al., 1992;Pan and Fesenmaier, 2006a;Pan et al., 2006;Richmond et al., 2003).
Internet is the cheapest and fastest method of getting information on any destinations. Tourists are adopting the internet as a major source of information for vacation planning (Lake, 2001;Pan and Fesenmaier, 2006a). In a qualitative research it is found that Malaysian tourists prefer to refer to the internet, friends and relatives since they consider these as reliable sources of information prior to a vacation overseas (Zaridah et al., 2005).
Demographic characteristics are significantly related to choices of information source (Luo et al., 2004). Besides textual and graphic information, tourist also looks for spatial information such as where is the nearest shopping mall from the hotel that they want to stay, what type of eateries are located within walking distance from where they are going to stay, among others (Pan and Fesenmaier, 2006b). This is consistent with one of the elements of naive geography which states that "people use multiple conceptualizations of geographic space" (Egenhofer and Mark, 1995). Naive geography is concerned with the formal world of geographic common-sense. It is used as the basis in Geographic Information System (GIS) model design developed for people who are unfamiliar with the system. Furthermore the statement that says "maps are more real than experience" (Egenhofer and Mark, 1995) shows that map which has interactive and effective query capabilities can become resourceful. Evaluations of online maps have been carried out by scholars to assess their usability in tourism sector. Focus on the evaluations varies. While Dickinger et al. (2008) evaluate online maps using tagger's performance, Zhang et al. (2012) focus online map service websites and Pontikakis and Twaroch (2006) propose schematic maps as alternative to maps without topographic elements. In addition, performance of Google Map as a tourist's map has also been evaluated. Results of these studies show that there is a need to offer an online GIS map that is able to produce accurate results for spatial information.
With the increasing number of people travelling around the world and the increasing demand for online geo-based information, reliance on the web-based information has also increased. Therefore, providing an excellent web-based Geographic Information System (GIS) on the internet will be a challenging task to those who wants to provide geo-based information to tourists.

WEB-BASED GEOGRAPHIC INFORMATION SYSTEM (GIS)
Web-based GIS, an extension of Geographic Information System (GIS), consists of four major system components which include client, web server with application server, map server and data server (Peng and Tsou, 2003). These components are integrated to process spatial data on the web.
Client is a place for users to interact with spatial features and carry out spatial analysis. Users can use this platform to perform request to the web server. When a request is posted the output will be displayed on the client side. Client-side scripting is used to produce dynamic html and as a result interactivity is increased between the client and the server. Among popular client-side scripting include JavaScript and VBScript. Browser plug-in is also used together with the scripts to enable user to view spatial data. This is only required when certain map server software is used. Java applet is another type of client software for displaying spatial data. It resides at the web server and is downloaded from the server and executes it at the client side (Peng and Tsou, 2003;Plewe, 1997;Tsou, 2004).
A web server and application server, on the other hand, perform vital function in responding and processing the client requests. Peng and Tsou (2003) identified several ways for a web server to respond to client's request. First, is by sending existing HTML document and secondly is by sending Java applets. Finally, the request is passed to another program and invokes other sub program. Application server on the other hand, acts as a medium between a web server and a map server. Its functions include establishing, maintaining, terminating communication between these two servers. In addition, it interprets clients' requests and passes them to the map server, manages concurrent requests and balances loads among map servers and data servers. Furthermore, it also manages the state, transaction and security of spatial data. The map server fulfils spatial queries, conducts spatial analysis, generates and finally, delivers the requested maps. According to Peng and Tsou (2003) the output of a map server can be in two forms. First, filtered feature data is sent to the client for further analysis. Secondly, simple images file in jpeg or png format is sent to client. A data server is the component that serves spatial and aspatial data. Such example is the SQL server. A data server acts as a medium between a map server and databases. Databases are made of relational database and geo database. It is normally allocated at a separate location from that of the map and web server. This is to ensure safety of the data (Peng and Tsou, 2003;Tsou, 2004).
Web-based GIS relies on client-server architecture which can be of two types (Peng and Tsou, 2003;Plewe, 1997;Takino, 2003;Tsou, 2004). One is thick client, thin server and the other is thin client, thick server. The former allows users to manipulate and process data easily and fast. GIS data is sent to the client and stored on the client side using applet. Once the network connection is disconnected, the data will be lost. On the other hand, with thin client, thick server, data processing is carried out entirely by the server. The effectiveness of rendering process depends on the efficiency of the server. Multiple requests for the same location in a map and multiple requests for the different locations in a map at the same time might hamper the performance of the server. Thus, developer of a webbased GIS needs to put effort in designing the scripting of handling requests (Nurul Hawani et al., 2005;Takino, 2003). This is to ensure not only fast and efficient, but also accurate response (Mata, 2007;Tsou, 2004).
Numerous studies have been conducted on enhancing server capabilities. Since its inception in the early 1990s with the introduction of Xerox Map Viewer, web-based Geographic Information System (GIS) or web-based GIS have been gaining popularity. Scholars now have seen the benefits of using web-based GIS in their discipline (Dragicevic, 2004). In one study, a web-based GIS system is implemented using server/client system in which image pictures are created one by one on GIS server machines when requested by a web client. In this system, data conversion and exchange occurs frequently on the server system. This results in difficulties to respond to web clients' requests within a short waiting time. A distributed data processing model for Web-based GIS by utilizing web client's hardware and software resources is then proposed by Takino (2003). This system transfers spatial index files which contain the structure and actual address of the spatial data on data site to clients' PCs when they access a web server. This model speeds up the processing capabilities of the server when multiple clients request for different spatial data sections. Tsou (2004) has integrated image processing tools in a web-based GIS model to assist in environmental monitoring and natural resource management. The model has enabled regional park rangers and local natural resource managers to utilize the capabilities of web-based GIS in monitoring and managing the resources. GIS software and remote sensing data are quite expansive for a non expert user to own. Furthermore, installing the software poses another hurdle to this user. Therefore, web-based GIS application can overcome these barriers. Tsou (2004) has managed to display satellite images in his webbased GIS model. To achieve this, three levels of GIS services which consist of data archive, information display and spatial analysis are combined together in the system architecture. However, Tsou (2004) did not proceed further in evaluating the system model in terms of its effectiveness and efficiency in retrieving the spatial data required. Kyem and Saku (2009), on the other hand, look at the use of web-based GIS in public participation within the local and indigenous communities. Potential benefits of a web-based GIS application in public participation are discussed in depth in this research. Even though this research only provides theoretical background, but it is an eye opener to other scholars to develop models to suit the needs. As a starting point, Kyem and Saku (2009) suggested the use of Google Earth to facilitate detailed observations and the creation of maps to address the community concerns about developments in their areas. Web-based GIS promotes online communication and interaction between community members beyond boundaries. Forums and group discussions are conducted online. This has cut down cost of space and travelling time. Furthermore, communication and interaction can be carried out 24/7. However, Kyem and Sake (2009) did not discuss on the methods of evaluating the web-based GIS model they suggested in terms of effectiveness and efficiency of the model. Sidlar and Rinner (2007) analyze the use of argumentation map as a decision support system tool. Based on a quasi-naturalistic case study, the study looks into the aspects of general usability of an argumentation map. By focusing more on learnability, memorability and user satisfaction with this tool's functionality, it is found that users are generally satisfied. In this research, additional components which consist of map navigation, display of discussion on contributions and online status of participants are also included. Even though this research contributes participatory spatial decision support systems to the knowledge, it does not look into the efficiency and effectiveness of the system.
Researches discussed above focus more on their models' usability rather than the effectiveness and efficiency of the systems. Therefore, it is the main intention of this research to integrate geographic information retrieval technique and evaluation method into web-based GIS research area in order to facilitate the performance evaluation process of the web-based GIS system.

TEST COLLECTIONS AND METHOD
Test collection in this study only covers tourist related information features in Shah Alam. A query list of 92 keywords was obtained from 31 bachelor Degree of Urban Studies and Planning Programme students in the University of Malaya. Relevant judgment was obtained from five experts who have known Shah Alam for more than 10 years.
Shah Alam is the capital city of Selangor, one of the states in Malaysia. Being one of the earliest planned city in Malaysia, Shah Alam has her own way of being distinctive from Kuala Lumpur, the capital of Malaysia. Besides housing the state's departmental offices and buildings, Shah Alam offers shopping for arts and crafts, traditional clothes and materials. There is not many traffic jams in Shah even though there are many cars on the road. Thus, this has encouraged people who are intolerant to traffic congestions to shop around here.
Spatial data in this collection consists of point and polyline features and is distributed into eight different layers. Point feature represents accommodation, attraction, eateries, Auto Teller Machines (ATMs), facilities and landmark. Likewise polyline feature represents railroad and street. Thus, test collections of Shah Alam City Centre are divided into eight layers. Each layer has attributes related to it. The attributes of each document constitute the aspatial data. These categories are in accordance to the study conducted by Plosker (2006) and Pan and Fesenmaier (2006a). Details of the documents are described in Table 1.
The two systems that are evaluated include single layer and multi layers web-based GIS systems. Both systems are of thin client, thick server. Single layer system is a web-based GIS system that requires a user to choose specific layer before a search function can be carried out. This system can only perform one request at a time. In addition, this system is case sensitive and use only exact match for results returned. This system reflects, as mentioned earlier, the current off the shelf web-based GIS products provided by several wellknown GIS developers. On the other hand, multi layers system is produced through customization of the one of the current off the shelf web-based GIS products. Slightly modified in terms of case sensitivity and layers integration, this system has the ability to perform multiple layers search function using a single step. In addition, this system is also able to carry out partial match query function. Finally, system testing is used as the main evaluation method for experiments. Precision and recall are the most known formal performance measurement based on systems returning a set of results in information retrieval (Clough et al., 2006;Martins et al., 2005). Information needs may vary from user to user in which some users may require high recall and low precision and vice versa (Salton and McGill, 1983). Usually at recall levels 0, 10, 20 and 30%, respectively the interpolated precision score is equal to 33%. As recall level increases, the precision level decreases (Baeza-Yates and Ribeiro-Neto, 1999). However, a good system is the one that can exhibit both a high recall and a high precision (Salton and McGill, 1983). Precision refers to the ratio of the total number of relevant documents retrieved by the sum of documents retrieved (Bucher et al., 2005;Martins et al., 2005;van Rijsbergen, 1979). It is represented by: where, rd = Total number of relevant documents retrieved nr = Sum of documents retrieved Likewise, recall is defined as the ratio of the total number of relevant documents retrieved to the sum of relevant documents [both retrieved and not retrieved] (Bucher et al., 2005;van Rijsbergen, 1979). It is represented by: In addition the F measure which also known as Harmonic mean, combines recall with precision and is usually used in problems when the negative results outnumber the positive ones. Thus, F measure equally weighs precision and recall and is given by: In this study, evaluation process of both web-based GIS systems is measured by precision, recall and F measure. Experiments are conducted using the Shah Alam City Centre's test collections. The scores collected from each experiment are then calculated and compared between the two systems; the single layer and the multi layers systems. Results are presented in the next section.

EXPERIMENTAL RESULTS
Results of evaluations carried out on Shah Alam test collections show that multi layers system (38%) produce higher results returned as compared to single layer system (25%). In addition, recall for multi layers system (0.9<r<1.00) has a higher and smaller range than the one produced by single layer system (0.10<r ≤1.00) which has a lower and wider range of recall score. Furthermore, the former system is able to produce a double percentage of full score of recall (98%) as compared to the latter system which is only 48%. Nevertheless, both systems are able to produce a perfect score of precision. This is important since tourist information needs to be precise and accurate in terms of providing geographical locations as discussed by Egenhofer and Mark (1995). F measures of both systems, however, show a big gap between the two. Similar to recall score, F measures for multi layers system (0.9<r<1.00) has a higher and smaller range than the one produced by single layer system (0.18<r ≤1.00) which has a lower and wider range of recall score. In addition, single layer system is only able to produce 48% of a perfect F measure score as compared to 93% by the multi layers system. Table 2 shows the evaluation results on Shah Alam City Centre test collections using the single layer system as compared to multi layers system. Results have proven that multi layers web-based GIS is able to perform better as compared to the single layer system. It has 100% precision and 98% recall. Thus, it is considered an efficient system in terms of producing required results as discussed by Salton and McGill (1983), Mata (2007) and Tsou (2004). However, the efficiency of this system is not based on time efficiency as discussed by Takino (2003). Three distinguish limitations of the single layer system can be drawn from this evaluation. First, is its case sensitivity? Any query keyword must match the one in its database. Second, it uses an exact match query function. This system is not able to handle partial match query. A query on 'attractions' will not return any results if its database stores the word 'attraction' only. Finally, it is tedious to take many steps in order to perform a single query. Users will definitely feel unhappy to use this system because of these reasons or system inabilities (Joachims et al., 2007). Therefore, enhancements using the multi layers approach to the single layer system are required to enable the system to perform better in the future.

CONCLUSION
Results have shown that multi layers system which consists of case insensitivity, multi layers integration and partial match query function, is required to yield better results. It has shortened the steps taken to perform search function of geographic related information by tourists as mentioned by Andreas and Volker (2007).
These findings have answered the main objectives of this research. Based on the above evaluations, it shows that there is a significant difference between precision, recall and F measure among the two systems. Results have proven that multi layers approach is able to produce the highest score in precision, recall and F measure. These findings have also contributed to the body of knowledge in the area related to web-based GIS or online maps and online information search by tourists which were previously conducted by scholars such as Pan and Fesenmaier (2006b), Dragicevic (2004), Dickinger et al. (2008), Zhang et al. (2012), Pontikakis and Twaroch (2006), Tussyadiah and Zach (2012) and Luo et al. (2004).
Although it performs better than the single layer system, multi layer web-based GIS approach has two limitations. First, is incorporating irrelevant document in its results return. Due to its ability in performing partial match query, this system is able to search and present results of a query that has a prefix and suffix of a word. For example the word 'art' can be a part of 'GoKarts' or 'department'. In this case, the results return is irrelevant even though it bears the query keyword. Second, is that it is unable to capture word that has similar meaning. For example the word 'eating' can be replaced by food or eateries in the system. This will enhance the system capabilities in handling query. Furthermore, results return according to the query keyword list is still below 40%. Therefore further enhancement needs to be carried out by scholars in the web-based GIS community.

ACKNOWLEDGMENT
This study was supported in part by the University of Malaya Research Grant (UMRG) under Grant No. FR212-2007A.