Ignorance is bliss? An empirical analysis of the determinants of PSS usefulness in practice

Planning support systems (PSS) enabled by smart city technologies (big data and information and communication technologies (ICTs)) are becoming more widespread in their availability, but have not yet been fully recognized as being useful in planning practice. Thus, a better understanding of the determinants of PSS usefulness in practice helps to improve the functional support of PSS for smart cities. This study is based on a recent international questionnaire (268 respondents) designed to evaluate the perceptions of scholars and practitioners in the smart city planning field. Based on the empirical evidence, this paper recommends that it is imperative for PSS developers and users to be more responsive to the fit for task-technology and user-technology (i.e., utility and usability, respectively) since they positively contribute to PSS usefulness in practice and to be more sensitive to the potential negative effects of contextual factors on PSS usefulness in smart cities. The empirical analyses further suggest that rather than merely striving for integrating smart city technologies into advancing PSS, the way that innovative PSS are integrated into the planning framework (i.e., how well PSS can satisfy the needs of planning tasks and users by considering context-specificities) is of great significance in promoting PSS's actual


Introduction
The rapid development of new digital information and communication technologies (ICTs) (e.g., Internet of Things sensors, artificial intelligence, networks) and big data in the realm of smart cities has opened up new opportunities for the development and application of planning support systems (PSS) (Barns, 2018;Geertman & Stillwell, 2020;Vallicelli, 2018).According to Pettit et al. (2018), PSS-as enabled through big data, city analytics, and modeling-provide potential benefits for smarter city planning that should be given consideration.
Here, PSS can be understood as geo-information technology-based instruments that are dedicated to supporting those involved in planning in the performance of their specific planning tasks (Geertman, 2006).Studies show that the potential benefits arising from new ICTs and big data to PSS are multidimensional.For instance, real-time and personalized (big) data concerning built environment (e.g., traffic flow, energy usage, public safety, and environmental protection) can be captured, analyzed, and integrated into various types of PSS because of the rapid development of electronic data sensors in augmenting city functions (Bettencourt, 2014;Geertman & Stillwell, 2020;Thakuriah, Tilahun, & Zellner, 2017).Urban planning-relevant spatial analyses are substantially increased with the advent of urban data analytics and ubiquitous computing (Babar & Arif, 2017;Rathore, Ahmad, Paul, & Rho, 2016).Besides, various smart ICTs (e.g., web-based platforms, online social networking, blogs, electronic voting, internet petitions) can also broaden and deepen political participation and collaboration in the planning field by enabling ordinary people to have access to the planning process (Khan, Ludlow, Loibl, & Soomro, 2014;Stratigea, Papadopoulou, & Panagiotopoulou, 2015).
Although new PSS, as enabled through big data, and new smart ICTs offer the potential for smarter city planning and are becoming more widely available, it should be noted that planning practitioners have never fully embraced the much wider diversity of available methods, techniques, and models developed in research laboratories and private companies (Geertman, 2006;Geertman, 2017;Pettit et al., 2018).For quite some time, there exists an implementation gap for PSS-that is, an apparent mismatch in planning practice between supply, demand, and applications of PSS and their outcomes (dedicated information and knowledge) (Brömmelstroet & Schrijnen, 2010;Geertman, Goodspeed, & Stillwell, 2015;Vonk, 2006).To shed light on the reasons for the PSS implementation gap, Vonk, Geertman, and Schot (2005) conducted systematic research and identified a wide range of bottleneck indicators, including human, organizational, and institutional, as well as technical factors, that have blocked widespread usage and adoption of PSS in planning practice.Based on their recommendations, research has been conducted to investigate the usefulness (or added value) of PSS in practice (Pelzer, 2015;te Brömmelstroet, 2013).It is highlighted that thorough research into the potential benefits of PSS can help arouse awareness among planners of the existence of PSS and of the purposes for which PSS can be used in a supportive way (Vonk, 2006).
However, studies indicate that in the actual application of PSS, a range of factors influence the usefulness of PSS.For instance, some authors argue that the quality of PSS functional support for planning tasks is decisive for PSS success (Geertman & Stillwell, 2009;Harris & Batty, 1993;Klosterman, 1997).Other authors claim that the perceived user-friendliness is positively related to the success of PSS (Pan & Deal, 2019;Russo, Costabile, Lanzilotti, & Pettit, 2015;Vonk, 2006).More recently, it has been widely accepted that PSS need to be enhanced to align the instruments more with the dynamic characteristics of planning processes since the specific situations or contexts in which PSS are embedded have a significant influence on how PSS work in actual planning practice (Champlin, te Brömmelstroet, & Pelzer, 2019;Geertman, 2006;Geertman, 2017).From this, it can be seen that increasing attention is being paid to the factors influencing PSS usefulness.However, there is a lack of comprehensive conceptual frameworks and empirical studies to systematically investigate the determinants (i.e., important success and failure factors) of the usefulness of PSS, even though Vonk et al. (2005) emphasized the necessity of such an effort in 2005.Fifteen years later, such elaborations seem to be much needed, since PSS are now being confronted with implementation challenges in the realm of smart cities (Pettit et al., 2018).
To improve the usefulness of PSS in actual planning practice, this paper aims at utilizing the knowledge of PSS as a benchmark to investigate the important success and failure factors determining the usefulness of PSS in the realm of smart cities.This study can be seen as an extension to the study of Vonk et al. (2005).Different from Vonk's study, which focused on bottlenecks blocking widespread acceptance of PSS, this paper examines the factors determining the actual use of PSS in planning practice.Consequently, the key argument made here is that unlike fifteen years ago when applications of PSS in practice were primarily experimental and tended to be less accepted, in recent years PSS enabled by smart city technologies have been more easily accepted and widely implemented into different planning practices, such as environmental planning, tourism planning, public health service, disaster management, etc. (see also Geertman & Stillwell, 2020).Thus, a better understanding of the factors influencing PSS's actual usefulness in practice enables us to provide effective and holistic solutions to the implementation gap of PSS and improve the support function of dedicated PSS in smart cities.
The next section describes the state of the literature with respect to PSS usefulness and then develops a conceptual framework, including the most important factors determining PSS usefulness as observed in the literature.The third section describes the research methodology.Section 4 elaborates on the empirical results and describes the important success and failure factors determining PSS usefulness.Section 5 reflects on the conceptual framework and empirical results, followed by the conclusion in Section 6.

Usefulness of PSS in smart cities
Cities are full of ubiquitous information technologies and they are increasingly understood as smart and connected urban areas (Batty, 2013).To turn the omnipresence of urban technological innovations into benefits for planners and citizens, they could be used within PSS which help make the planning process more efficient and handle complexity better (Geertman & Stillwell, 2020;Pettit et al., 2015).Recently, a growing body of research seeks to better understand how PSS can make use of these new ICTs and data sources to support the planning, management, and implementation of a smart city.
First, some authors argue that the rise of a smart city leads to an exponential increase in data by several orders of magnitude; consequently, such enormous volumes of data or big data act as valuable input for PSS (Babar & Arif, 2017;Bettencourt, 2014).By exploring the ways in which this considerable amount of real-time and very up-todate data collected through various sources are linked using data-driven analytic PSS, valuable information and knowledge can be produced for service and administration purposes (e.g., crowd sensing-based traffic measurements) (Barns, 2018).
Second, the emergence of big data and new ICTs generates sophisticated data analytics and geospatial modeling, which further helps expand the scale and scope of PSS applications (Khan, Anjum, Soomro, & Tahir, 2015;Rathore et al., 2016;Thakuriah et al., 2017).Traditionally, PSS are mainly accepted in limited fields like transportation planning and expertimental research.However, recent studies indicate that PSS enhanced by real-time data and new ICTs are increasingly implemented to address a wide spectrum of urban issues such as resource and environmental management, basic farmland protection, tourism planning, housing planning, etc. (Geertman & Stillwell, 2020).
Third, PSS can also be used to facilitate technology-mediated interaction between the civil society sphere and the formal political sphere and broaden access to smart city planning processes (Lock, Bednarz, Leao, & Pettit, 2019;Panagiotopoulou & Stratigea, 2017;Saad-Sulonen & Horelli, 2010;Zhang, Geertman, Hooimeijer, & Lin, 2019).By offering new opportunities for more direct and convenient citizen access to the planning process of smart cities and including a broader range of new perspectives, ideas, opinions, and knowledge, it helps governments and its agent planners to gain insight into the views of other stakeholders and avoid pitfalls caused by unawareness of the specificities of individuals and communities (Geertman et al., 2015;Panagiotopoulou & Stratigea, 2017).
As Geertman et al. (2015) highlight, the integration of new big data and ICTs into PSS in the era of smart cities not only have the capability for collecting, managing, analyzing, and storing information about cities more efficiently than before, they also present planners and managers with opportunities to draw on this information to improve city life.It is proclaimed that this trend will continue over the coming years, particularly given the rapid development of ICTs (Barns, 2018;Vallicelli, 2018).However, it should be noted that although PSS offer the potential to harness the power of urban big data and new ICTs and digital tools to support smart city planning, the usefulness of PSS for smart cities is weakened due to restraints related to PSS implementation (Pettit et al., 2018).A lot of PSS are developed by academic researchers, but the tools are ultimately not responsible for planning decisions (Brömmelstroet & Schrijnen, 2010).In practice, low PSS education and training and low technical skill are highlighted to influence PSS usefulness (Pelzer, 2015).Besides, some of the PSS cannot achieve a balance of complexity and simplicity due to a lack of flexibility and transparency (te Brömmelstroet, 2012).As Geertman (2017) criticizes, although PSS application studies actually apply PSS, most intended PSS applications are not realized.Based on this, we argue that to strengthen and optimize the transformative potential of PSS in smart cities, factors determining PSS usefulness should be systematically investigated.

Factors influencing PSS usefulness
The usefulness of PSS refers to the positive influence a PSS can have on practice (Pelzer, 2015).te Brömmelstroet, 2013 highlights that PSS usefulness can be measured at the process or outcome level.In several empirical cases studies, PSS have been identified as useful for planning practice by helping the public to express their needs, promoting interpersonal dialogue and debate, producing information in a form which can be understood and used by the 'non-specialists', and visualizing and interpreting keyword data (Goodspeed, 2016;Pelzer, Geertman, van der Heijden, & Rouwette, 2014;te Brömmelstroet, 2012;Zhang et al., 2019).
Based on Nielsen (1993), Pelzer (2017) reveals that two main factors-utility and usability-can be identified to determine the usefulness of PSS (Fig. 1).Here, "utility is the question of whether the functionality of the system in principle can do what is needed" (Nielsen, 1993:24).According to Pelzer (2017), "'do what is needed' refers to the effect on the planning tasks a PSS is intended to support in the context of PSS".The concept of 'task-technology fit' is applied to make sense of utility (Goodhue & Thompson, 1995;Pelzer, Arciniegas, Geertman, & Lenferink, 2015).It assumes that only if the characteristics of the PSS fit the planning task, can the utility of PSS be fulfilled.Table 1 indicates the commonly used utility indicators that have a significant influence on PSS usefulness.
Usability, then, is about 'how well users can use that [utility] functionality' (Nielsen, 1993:25).'How well' indicates the user experience of using a PSS.According to Russo et al. (2015), usability is widely acknowledged within the Human-Computer Interaction literature as user-technology fit (Vonk, 2006), which focuses on the effectiveness and efficiency of the interaction between user and system (PSS), the user engagement and the derived satisfaction.Normally, a higher level of usability of a PSS tool can improve the acceptance of a PSS in practice.Table 2 indicates the commonly used usability indicators that have a significant influence on PSS usefulness.
te Brömmelstroet (2010) finds that a technical focus is insufficient to improve the PSS added value, since key bottlenecks of the use of PSS are actually centered on 'soft issues' like poor connections to the planning process.Hence, he argues that usability of PSS should be improved to link the instruments more with the dynamic features of users and planning issues.According to Vonk and Ligtenberg (2010), since social activity is often dynamic and nuanced, knowledge often needs to be contextualized to be useful in planning.This means the actual use of PSS should not be reduced to a rational and linear process or one-size-fit-all approach; instead, PSS should become more aware of situational specificities in which a PSS is embedded.A systematic review of contextual factors influencing the usefulness of PSS was conducted by Geertman (2006).In his elaborations, six contextual variables were identified to influence the potential planning support roles of PSS.Table 3 shows the different kinds of contextual variables and explanatory indicators in the PSS research.However, to the best of our knowledge, no systematic empirical study has been conducted to examine the importance of these contextual variables in shaping PSS usefulness.

Towards a conceptual framework
According to Pelzer (2017:93), "the best way to understand usefulness is to integrate the frameworks of Geertman (2006) and Nielsen (1993)".Based on his work, this paper makes such an attempt by integrating the context dimension into the usefulness model to be able to better understand the factors determining PSS usefulness (Fig. 2).Therein, the task-technology fit indicates to the appropriateness of the technology to handle the task at hand; while the user-technology fit indicates to the goodness of fit between the capabilities of the user and the functionalities offered by the technology.
This adapted model illustrates that PSS usefulness is mainly explained and influenced by the utility and usability factors.When being implemented into practice, context then plays a crucial role in affecting the usefulness of PSS.It should be noted that the task-technology fit and the user-technology fit are also highlighted as part of the framework to help us understand better the meaning of utility and usability.In the following sections, the elaborated utility indicators, usability indicators and contextual indicators were applied to thoroughly investigate the determinants (i.e., important success and failure factors) of PSS usefulness in practical smart city projects.

Data collection
This study is mainly based on an international questionnaire which was undertaken over the course of three months, from May to July 2019.The aim of the survey was to collect in-depth information about the application of PSS in practical smart city projects.The questionnaire was mainly headed to the research community of Computers in Urban Planning and Urban Management (CUPUM).The reason for selecting the CUPUM community as survey respondents is that CUPUM has been one of the major international academic platforms to discuss the latest ideas and applications of computing technology, aiming to address a diverse range of social and environmental issues that would affect urban planning and development based on computing technology.2Thus, CUPUM members are normally considered to be equipped with comprehensive and detailed knowledge and specialized skills in terms of PSS, which enables them to understand the central goal of the survey: to gain knowledge to better understand the opportunities and threats of planning support technology in computational urban planning for smart cities.By using electronic and regular mailing lists, approximately 1300 people around the world were invited to participate in the survey.
In the web-based survey, forty-five statements constitute the main part of the questionnaire, categorized for types of urban problems, stakeholders, utility, usability, added value, and context.Except for the statements linked with this study, questions to decide the features of the survey population (e.g., gender, age, profession, origin, expertise with planning support ICTs) were also attached.The statements were based on the previously identified PSS literature.For each of the statements, a seven-point scale (from 1 (low) to 7 (high)) was offered to the respondents.Respondents were specifically asked whether or not they have been-academically and/or professionally-involved in smart city projects over the past years.The follow-up questions specifically addressed their involvement in such projects.In this paper, the 28 statements concerning utility, usability, and context were applied to do the analysis.

Data analysis
The questionnaire data was analyzed based on Fig. 2. First, the analysis only included respondents who were involved or are currently involved in smart city projects, because this real-life experience gives them the actual possibility to evaluate the statements.Furthermore, in the questionnaire we asked the respondents about their level of expertise in planning support technology and if they have practical experience in working with this technology in real-world projects.Second, the scores of utility variables, usability variables, and contextual variables were calculated to explore the success and failure factors.More specifically, for both the utility and usability statements the success factors were calculated by combining the frequency scores of the answer categories 'successful' and 'very successful'; and for the context statements by combining the frequency scores of the answer categories 'positive' and 'very positive'.Likewise, for both the utility and usability statements the failure factors were calculated by combining the frequency scores of the answer categories 'unsuccessful' and 'extremely unsuccessful'; and for the context statements by combining the frequency scores of the answer categories 'negative' and 'extremely negative'.For clarification reasons, the frequency scores show in absolute or relative sense how many participants select a specific sevenpoint category.For instance, among the 268 respondents, 155 respondents selected the category 'successful' or 'very successful' in the 'spatial analysis' indicator; as a result, the frequency score is 155 in absolute sense and 58% in relative sense (absolute score / total population of respondents = percentage; 155/268 = 58%).
Third, to further validate the results, sub-groups of respondents were examined and compared with the results from all respondents.Subgroups were distinguished on the basis of geographical origin/ economic development level and profession.First, all respondents were categorized as respondents from China or as respondents from North America, Europe, Japan, and Australia (abbreviated as NEJA).This classification is mainly due to three major reasons: 1) more than half of the questionnaire respondents appeared to be from China, which is foremost attributed to the fact that the 2019 CUPUM conference was organised in Wuhan, China (see Fig. 3a); 2) according to some authors, China, North America and Europe have the largest group of smart cities projects (Jiang, Geertman, & Witte, 2019;Zubizarreta, Seravalli, & Arrizabalaga, 2016); thus, research on the application of PSS in China and NEJA would help understand the strengths and limitations of PSS in the workplace; 3) China and NEJA countries are at different levels of economic and societal development, thus posing different challenges for PSS usefulness in practice.Second, all respondents were categorized based on their professions: respondents from academia on the one side and respondents from practice on the other.Respondents from academia include academic researcher/scholar and doctoral students, while respondents from practice consist of planners, designers, and politicians (see Fig. 3(b)).
Finally, the variables were interpreted in relation to the conceptual framework, thus providing insight into the important factors that contribute to the success and failure of PSS usefulness in practice.The average frequency score per group of important success and failure factors was measured, which helps to derive the relative importance of the different factors in the conceptual framework.From this measurement, an overall picture of determinants of the usefulness of PSS in the realm of smart cities can be clearly illustrated.

Usability indicators Description Transparency
The extent to which the underlying models and variables of the PSS are visible to users User friendliness The extent to which participants are able to use the tool themselves Interactivity The extent to which the tool can directly respond to the users' questions and suggestions Flexibility The extent to which the tool can be applied to different planning tasks Calculation time The time participants have to wait before an analysis is completed Data quality The extent to which the input data is considered valid Reliability The extent to which the outcomes of the tool are considered reliable

Utility indicators Description
Geo-data gathering Functional support for geo-data collection Geo-data storage Functional support for geo-data storage Visualization Functional support for creating images, diagrams, or animations One-way informing Functional support for transferring information in one direction only Two-way communicating Functional support for facilitating communication and discussion between those involved in planning through supporting flow of planning related information between them (e.g., Touch Table) Spatial analysis Functional support for examining spatial patterns of human behavior Spatial modeling Functional support for simulating spatial objects or phenomena Spatial designing Functional support for idea design and drawing Scenario building Functional support for identifying possible "realities" of the future Impact analysis Functional support for determining the potential consequences of a plan

Exploration of responses
Analysis shows that 268 respondents have filled out the questionnaire, which is estimated as approximately a 20.6% response rate, which is a good result for a web-based international inquiry.Among the 268 respondents, 175 respondents are involved in smart city projects in which PSS have played an important role.Fig. 3(a) reveals that 93 out of the 175 respondents (53%) are from China, 58 of the respondents (33%) are from NEJA (North America, Europe, Japan and Australia) and 24 (14%) are from other countries (mainly Russia, South Africa, Brazil, and India).Fig. 3(b) indicates the profession of the respondents.It illustrates that the majority of the respondents are academic researcher/scholars (72 respondents; 41%) or planners (58 respondents; 33%) whereas only 9 respondents (5%) are designers or politicians.Besides, 37 doctoral students account for 21% of the total respondents.As mentioned in the previous section, designers, politicians and planners are combined and categorized as the practitioner group and academic researcher/scholars and doctoral students are treated as the academic group.indicators-'geo-data gathering', 'impact analysis', 'one-way information', 'geo-data storage', 'scenario building', 'spatial designing'.As noted, nine out of the 12 important success indicators are related to PSS utility and three are related to usability.This confirms the finding that the quality of support functions along with its user experience are decisive for PSS usefulness (Vonk & Ligtenberg, 2010).Additionally, it should also be noted that no contextual indicators are considered to be important success indicators.

Respondents from China and NEJA
Analysis of respondents from China and NEJA shows that the importance scores of success and failure indicators are consistent with the scores of the total.Thus, the 12 important success indicators and the 12 important failure indicators in China and NEJA are further analyzed and compared with the total.As mentioned by Vonk et al. (2005), a small difference between the results for subgroups and the general results would indicate unanimity, which would contribute to the validity of the general results, whereas larger differences would indicate the opposite.In our study we have applied Chi-Square tests to determine whether there is a statistically significant difference between subgroups in terms of success and failure factors.
Fig. 5 (a) indicates the scores of the 12 important success indicators distinguished earlier which scored high on importance in total (over 30%), compared with the scores from subgroups of respondents based on geographical origin / economic development level.The subgroups consist of 93 respondents from China and 58 respondents from NEJA.It shows that the difference in some indicators between China and NEJA are marginal (namely 'visualization', 'geo-data gathering', 'transparency'); for all other indicators the outcomes differ substantially.Further analysis shows that indicators gaining higher scores in NEJA include two usability indicators ('reliability' and 'data quality') and four utility indicators ('spatial modeling', 'impact analysis', 'one-way informing' and 'scenario building').Only the scores of 'spatial designing' and 'spatial analysis' in China is distinguished much higher.This could indicate that utility and usability indicators in NEJA are more conducive to the success of PSS implementation than their China counterparts.However, in statistical hypothesis testing (Chi-Square test), p-value (=0.0854) was reported larger than 0.05, indicating that the difference between China and NEJA in terms of success factors is not statistically significant.30%), compared with the scores from China and NEJA.In accordance with the total respondents, failure indicators are also chiefly linked with contextual indicators in both China and NEJA.Then, some indicators (i.e., 'technical skill', 'adaptability to user need', 'user attitude', 'political system' and 'political pressure') in China distinguish themselves from their NEJA counterparts by quite a margin, indicating the stronger negative effects of context on PSS usefulness in China.Despite the observed difference between China and NEJA in terms of failure factors, the significance was not determined by the Chi-Square test as pvalue (=0.3270) was reported larger than 0.05.

Academic respondents and practitioners
Fig. 6 (a) indicates the scores of the 12 success indicators distinguished earlier which scored high on importance in total (over 30%), compared with the scores from subgroups of respondents based on profession.The subgroups are made up of 108 academic respondents and 67 practitioners.Comparative analysis reveals that the scores of some utility indicators-'spatial analysis', 'spatial modeling', 'geo-data gathering' and 'spatial design' and 'geo-data storage'-are distinguished much higher in the practitioner group than their academic counterparts whereas other indicators between the two groups differ little from each other.Then, both groups see 'visualization' and 'spatial analysis' as the most important success factors.This indicates the high satisfaction of both subgroup respondents in using PSS for analysis and visualization.In statistical hypothesis testing, p-value was reported much larger than 0.05 (p-value = .8412),showing that the difference between practitioners and academic respondents in terms of success factors is not significant.It also means that success factors between practitioners and academic respondents show a high consistency.
Fig. 6 (b) illustrates the scores of the 12 important failure indicators distinguished earlier which scored highest on importance in total (over 30%), compared with the scores from academic respondents and practitioners.In general, the scores of the 12 important failure indicators in the practitioner group are much higher than their academic counterparts.This indicates that while practitioners are more sensitive to failure factors than for instance academia, this can negatively contribute to the PSS-implementation gap.Besides, it should also be noted that the only failure indicator that had a higher score for academic experts than for practitioners is 'two-way communication', showing that academic experts are more affected by the quality of communicating functionality.In statistical hypothesis testing, the p-value (=0.0010, less than 0.05) shows that the difference between practitioners and academic respondents in terms of failure factors is statistically significant.Average score analysis has revealed that the influence of failure factors in the group of practitioners is much stronger than its academic counterpart.
Therefrom, further analysis was made to compare the scores of the success and failure indicators from subgroups of practitioners with different levels of experience in using PSS-experienced practitioners and less-experienced practitioners.Firstly, in terms of success factors, the scores of the majority of utility and usability indicators are much higher for the group of experienced practitioners than for its less-experienced practitioner counterparts, while both have low scores for contextual indicators.In statistical hypothesis testing, the p-value (< 0.0001, much smaller than 0.05) determined this difference significant.Secondly, in terms of failure factors, both groups gave high scores for contextual indicators, however, the scores of experienced practitioners are relatively higher than of its less-experienced practitioner counterparts.The Chi-Square test has also determined the statistically significant difference between these two groups (p-value < .0001,much smaller than 0.05), which verifies the obtained results.

Interpretation of results
The ranking of success and failure factors in previous analyses indicates that a wide diversity of factors is considered to be important.But the relations among those factors are not explicit from such a list.Therefore, our theoretical framework on determinants of PSS usefulness is used to interpret these relations and help to understand the added value of the results.Fig. 7 shows the theoretical framework, with the important success and failure factors and their scores incorporated within the three key factors (utility, usability, and context) in total, compared with two subgroups based on geographical origin/economic development level and profession.The percentages were calculated as an average from the relative frequency scores of the success or failure indicators, as shown in Figs. 4 to 6.For instance, the percentage of 'utility success factors in NEJA' was obtained by calculating the average of the scores of the nine utility indicators in Fig. 5a.
The earlier notion that there is not a single success and failure factor influencing the usefulness of PSS for smart cities, but quite a diversity, is clearly conveyed by previous analyses.Fig. 7 (black bars) shows that factors related to utility, usability, and context can either be success factors or failure factors.In general, the scores of the utility and usability success factors (39% and 38%, respectively) are higher than their failure factor counterparts (23% and 0%, respectively).This means the positive effects of utility and usability factors on PSS usefulness outweigh their negative effects.More detailed analysis shows that except for 'two-way communication', almost every utility indicator is among the important success factors, indicating that PSS utility constitutes the main determinant of success for PSS usefulness.However, the support function related to communication and discussion between those involved in planning (i.e., low score of 'two-way Then, usability success factors are mainly linked to the quality or characteristics of PSS in transforming input into output information production (i.e., high scores of 'reliability' and 'data quality' and 'transparency') whereas other usability success indicators gain moderate scores.Besides, no important usability failure indicators were identified.This confirms the statement by Pelzer (2017) that "usability has increased significantly over the last decade" and is widely treated as a necessary condition but not a sufficient condition for the success of a PSS tool.Third, it should be noted that all the contextual indicators were identified as important failure indicators.Thus, a declaratory judgment is made that context constitutes the main determinant of failure of PSS usefulness in practice for contemporary smart cities. Fig. 7 also shows the important success and failure factors in subgroups of respondents based on geographical origin/economic development level (dark grey) and profession (light grey).Some similarities and differences exist between the subgroups.Fig. 7 (dark grey) reveals that in terms of the total average scores of success factors, the NEJA group in general is more outspoken than its China counterparts, whereas concerning the total average scores of failure factors, China shows more vulnerability.However, previous analyses indicate that the difference between China and NEJA is not statistically significant, which indicates that key factors for success and failure between China and NEJA show a high consistency.Then, Fig. 7 (light grey) reveals that the total average score of success factors in the practitioner group is much higher than for its academic counterpart.The practitioner group shows a higher average score of failure factors than its academic counterparts, negatively influenced by contextual factors.In statistical hypothesis testing, between practitioners and academic respondents only the difference in terms of failure factors was determined significant.
In brief, similarities and differences exist between subgroups but results for the subgroups agreed, in general, with the results obtained earlier for the total respondents for both the important success and failure factors.This confirms the general validity of the results.

Reflections
This section further reflects on the conceptual framework built in Section 2 and discusses the extent to which the empirical results obtained in the realm of smart cities agreed with and/or differed from PSS studies in the literature.

Reflection on the conceptual framework
The results show that the theoretical framework proposed in Section 2 is helpful to examine the success and failure factors determining the usefulness of PSS in the realm of smart cities. Different from previous studies (Pelzer, 2017;Pelzer et al., 2014;te Brömmelstroet, 2012), our conceptual framework treats the contextual factor as an integral part of influencing the usefulness of PSS.As Pelzer (2015) argues, although task-technology fit (utility) and usability are valuable to understand PSS usefulness, the complexity of the task itself and a user's experience of a task could influence the evaluation of PSS usefulness.By building a more comprehensive and integrated framework, it indicates that the usefulness of PSS in the planning of smart cities is not achieved just by the PSS themselves but depends more on the different kinds of factors influencing the use of PSS in practice.
Despite the advantages the framework offers, there are some issues still being debated.For instance, in the conceptual framework, some usability indicators such as communicative value, integrality, and level of detail are not considered because of some overlap between utility and usability (e.g., communicative value as a usability indicator and communicative support capabilities as part of utility) (Pelzer, 2017) and inexplicable semantics.In addition, Bressers (2009) shows that contextual variables consist of two main levels-wider context (e.g., problem context, political context, economic context, cultural context, technological context) and structural context (e.g., policy style, networks & actors, strategies & instruments).It should be noted that not all contextual variables of these two levels are considered in this paper since we considered some contextual variables (e.g., economic, cultural) to merely influence the widespread acceptance of PSS (see Vonk et al., 2005).Despite the identified limitations of the framework, this framework arguably contributes to integrating the frameworks of Geertman (2006) and Nielsen (1993), as strongly recommended by Pelzer (2017), to build an effective model for studying the determinants of the usefulness of PSS in the realm of smart cities.

Reflection on the empirical results
The results obtained based on a large-scale survey make a good response to the current PSS research, but meanwhile goes beyond these research inputs.In the literature, PSS are found successful in exhibiting information in forms that are easy to understand by non-specialist users, facilitating interpersonal communication, displaying relevant scenarios, and helping the public to express their interests (Champlin et al., 2019;Pelzer et al., 2014;Zhang et al., 2019).In this paper, the results confirm that PSS utility (or functionalities) are the major success factors contributing to PSS usefulness.Some studies reveal that poor user-friendliness and interactivity have a negative impact on PSS usefulness (Pan & Deal, 2019;Russo et al., 2015).Results in this paper, however, show that usability in general is not considered an important failure factor.
Besides the aforementioned results, the authors have also found that the usefulness of PSS is associated with the types of urban problems that the users attempt to solve.3It shows that utility indicators concerning 'analyzing' and 'data processing' (e.g., 'spatial analysis', 'visualization', 'geo-data gathering', and 'geo-data storage') gain high scores in dealing with 'transportation & mobility' problems.Conversely, the scores of utility indicators concerning 'informing & communicating' and 'designing' (e.g., 'two-way communication' and 'spatial designing') are low in dealing with 'environmental', 'housing' and 'economic' problems.The different success and failure factors determining PSS usefulness within different urban problems are mainly caused by the interaction strength between functionalities and urban problems-that is, the extent to which PSS functionality fits to the task.
Still, the authors recognized the limitations of the results.Because of our selection procedure, the opinions and attitude from the group of citizens are not considered in this paper.According to some authors, however, ideas and knowledge from civil society can effectively promote the advancements of PSS and accelerate growth in participatory urban planning (Geertman & Stillwell, 2020;Pelzer, 2015;Zhang et al., 2019).Notwithstanding this limitation, the paper does not consider this to be detrimental to the validity of the obtained results since the perspectives from scholars and practitioners still provide a professional overview of determinants of success and failure of PSS in practices of contemporary smart cities.

Conclusions
Planning support systems (PSS) enabled by smart city technologies are becoming more widespread in their availability, but have not yet been fully recognized as being useful in planning practice.This paper extends and updates the work by Vonk et al. (2005) and aims to investigate and analyze the factors influencing PSS usefulness during the process of PSS being actually used in the realm of smart cities.Based on an international questionnaire, empirical evidence shows that 1) utility (explained by 10 indicators) constituted the primary reason for the success of PSS usefulness in practice; 2) context (explained by 11 indicators) primarily acted as a failure factor for PSS usefulness; and 3) usability (explained by 7 indicators) were identified as a necessary but not sufficient factor to achieve PSS usefulness.
In general, this study offers a comprehensive picture of the important success and failure factors determining the usefulness of PSS in the realm of smart cities.What can be deduced is that the factors that contribute to the success of PSS usefulness are not necessarily the same as the factors that contribute to its failure.This points to the idea that the implementation of PSS should take into account both sets of factors of avoiding failure and on ensuring success.Thus, this paper recommends that it is imperative for PSS developers and users to 1) be more responsive to the fit of task-technology and user-technology (i.e., utlity and usability, respectively) since they positively contribute to PSS usefulness in practice; and 2) be more sensitive to the potential negative effects of contextual factors on PSS usefulness in smart cities.
Finally, Vonk and Ligtenberg (2010) argue that intense cooperation with users to improve the analysis of planning tasks and user needs in specific contexts is promising as a means to enhance PSS use in planning practice.Therefore, the results obtained in this paper further suggest that rather than merely striving for integrating smart city technologies into advancing PSS, the way the innovative PSS are integrated into the planning framework (i.e., how well PSS can satisfy the needs of planning tasks and users by considering context-specificities) is of great significance in promoting PSS's actual usefulness.The largescale survey and empirical evidence acquired in this paper have provided valuable insights into realizing the full benefits of available PSS in smart cities. Future research could address these topics based on detailed case studies.Such a study helps discern context-aware determinants of the usefulness of PSS in specific smart city projects.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Fig. 4 .
Fig. 4. (white bars)  shows the success indicators with their importance scores (percentage of 'successful' and 'very successful' in the utility and usability statements and percentage of 'positive' and 'very positive' in the context statements) derived from the 28 statements.A wide range of indicators are significant.Illustrative in this respect is that 12 out of the 28 indicators gain scores of more than 30%.The three most important success indicators are 'spatial analysis', 'visualization' and 'spatial modeling', which are all related to PSS utility.Following these three utility indicators are three usability indicators-'transparency', 'data quality' and 'reliability'.Besides, six other indicators representing PSS utility are mentioned as important indicators-'geo-data gathering', 'impact analysis', 'one-way information', 'geo-data storage', 'scenario building', 'spatial designing'.As noted, nine out of the 12 important success indicators are related to PSS utility and three are related to usability.This confirms the finding that the quality of support functions along with its user experience are decisive for PSS usefulness(Vonk & Ligtenberg, 2010).Additionally, it should also be noted that no contextual indicators are considered to be important success indicators.Fig.4(blackbars) shows the failure indicators with their importance scores (percentage of 'unsuccessful' and 'very unsuccessful' in the utility and usability statements and percentage of 'negative' and 'very negative' in the context statements) derived from the 28

Fig. 4 (
Fig. 4. (white bars)  shows the success indicators with their importance scores (percentage of 'successful' and 'very successful' in the utility and usability statements and percentage of 'positive' and 'very positive' in the context statements) derived from the 28 statements.A wide range of indicators are significant.Illustrative in this respect is that 12 out of the 28 indicators gain scores of more than 30%.The three most important success indicators are 'spatial analysis', 'visualization' and 'spatial modeling', which are all related to PSS utility.Following these three utility indicators are three usability indicators-'transparency', 'data quality' and 'reliability'.Besides, six other indicators representing PSS utility are mentioned as important indicators-'geo-data gathering', 'impact analysis', 'one-way information', 'geo-data storage', 'scenario building', 'spatial designing'.As noted, nine out of the 12 important success indicators are related to PSS utility and three are related to usability.This confirms the finding that the quality of support functions along with its user experience are decisive for PSS usefulness(Vonk & Ligtenberg, 2010).Additionally, it should also be noted that no contextual indicators are considered to be important success indicators.Fig.4(blackbars) shows the failure indicators with their importance scores (percentage of 'unsuccessful' and 'very unsuccessful' in the utility and usability statements and percentage of 'negative' and 'very negative' in the context statements) derived from the 28

Fig. 5
(b) illustrates the scores of the 12 important failure indicators distinguished earlier which scored highest on importance in total (over

Fig. 5 .
Fig. 5. (a) Important success factors in China and NEJA and (b) important failure factors in China and NEJA.

Fig. 6 .
Fig. 6.(a) Important success factors in the academic group and the practitioner group (b) important failure factors in the academic group and the practitioner group.

Fig. 7 .
Fig. 7. Important success and failure factors influencing PSS usefulness in total (in black), between subgroups of respondents based on geographical origin/economic development level (in dark grey) and profession (in light grey).