Blockchain implementation in smart cities – discussion on performance indicators

Background: Blockchain project implementation in smart cities represents a novel challenge in information technologies. Lack of functional framework and guidelines impact these implementations and add additional uncertainty. Authors in their research on this subject seek to discover a functional framework to improve the implementation process. The research is divided into 3 parts: identification of best application areas, critical success factors, and implementation performance indicators. Purpose: The goal of this paper is to present findings on the performance indicators and offer additional insight into their nature as well as point to lesser-known performance indicators of blockchain projects. Study design/methodology/approach: Authors used the Delphi technique and according to the methodology, a panel of 33 experts was presented with a list of performance indicators and asked to add additional performance indicators as well as to assess their importance. The research foundation was based on the literature review which resulted in 9 performance indicators for the blockchain project implementation in smart cities. Findings/conclusions: The experts proposed 8 additional performance indicators; however, only 1 of them had required consensus to be accepted by the panel. This paper will disclose both groups of performance indicators and preserve them for further scientific discussion since the performance indicators that were proposed by the individual experts but did not reach panel consensus will not be contained in the furthe r research results. After all rounds of the Delphi method were finished the experts concluded the list of the 5 most important performance indicators for the implementation of blockchain projects in the smart city. Limitations/future research: Due to limited resources and lack of experts, the size of the panel is limited to only 33 experts. In addition, smart cities are usually managed by the public sector, so access to all data is limited. The findings presented in the paper can be further used to improve the efficiency of blockchain projects.


Introduction
The main topic of this article is performance indicators of blockchain project implementation in smart cities. Expertise on this topic is in high demand; however, there are very few available texts and studies on this topic in both scientific literature and other sources. The claims in the scientific literature are mostly based on literature reviews and individual case studies. To the author's O N L I N E F I R S T STRATEGIC MANAGEMENT, Vol. xx (20xx), No. x, pp. 0xx-0xx best knowledge, although available papers are peer-reviewed and produced through the valid application of the scientific method, few of them are based on the systematic study of multiple cases to use induction to gain additional insights. Most of the scientific studies on this topic deal with a single-use case, which can prove to be a limiting factor to case conclusions. The hype caused by the rise of cryptocurrencies propelled blockchain to become an overnight buzzword and a technology that gains users faster than the internet at the beginning of the 21 st century. The hype was caused by the sharp cryptocurrency price rise and the acceptance of Ethereum cryptocurrency, its underlying technology, and the ecosystem from the major industry players (Enterprise Ethereum Alliance). Suddenly during the blockchain explosion in the year 2017 th to exploit this trend, everything became "blockchain", not through the real well-designed application of blockchain technologies, but through the simple marketing trick of creating "blockchain this and that" or even simple adding blockchain to the name without real application of the blockchain technologies. Some of these projects were scams or simply unsuccessful. The downside of this trend created the wrong image of blockchain in both financial, political, and individual views. Blockchain wrongly become a synonym for cryptocurrencies (primarily bitcoin) and similar financial applications of distributed consensus peer-to-peer networks. This strengthened the claims in the general and scientific community that blockchain (bitcoin in particular) is "a new technology that can be used by criminals for money laundering" (Möser, 2013). The novelty of the technology and uncertainty created a veil that for almost 10 years prevented government and large business stakeholders from adequately precipitating the true potential of these technologies and their possible positive impact on information systems and challenges these technologies can solve. From the creation of Bitcoin in 2008 (Nakamoto, 2008) until the creation of EEA (Enterprise Ethereum Alliance) in 2017, blockchain technologies were mostly used and perceived through their financial aspects. After the creation of EEA, blockchain finally became the focus of the scientific community; however, this area of study was much undeveloped -only a few scientific studies on blockchain use in information systems existed before the year 2017. The authors also want to point out that the public incorrectly interpreted blockchain as a technology that can only be applied in the fintech sector (cryptocurrencies and banks) without realizing the other possible applications in the real and government sector. In the author's opinion, the high demand resulted in numerous scientific studies that emerged after 2017, however, articles substantiated by real-life data and use cases are still rare. The scientific community wanted to research a "novel topic" but real data was hard to find. Distributed nature of blockchain systems also makes experimental research simulating real-life use difficult because it is hard to simulate hardware and electricity extensively distributed peer-to-peer networks in laboratory conditions. Lack of tools, expertise, and knowledge limited the available possibility for research. Also, not many projects were implemented; therefore real data and knowledge were scarce. Time passed and implemented pioneer use cases of blockchain technology showed the added value and real benefits of their application in government and smart city environments (Xie, 2019), (Bhushan, 2020), (Khanna, 2021). These pioneer implementations offered a good starting point for further study. The goal of this paper is to show all steps and gained information from the author's study of blockchain technologies implementation in smart cities. Our research on the topic of blockchain implementation in smart cities is still ongoing and it is divided into 3 separate parts: identification of best application areas, critical success factors, and implementation performance indicators. This paper will only deal with a discussion on performance indicators of blockchain implementation projects in smart cities. This research uses the Delphi method.

Research topic and context -Application of blockchain technologies in the smart city
When the advantages of blockchain technologies are compared to the significant issues and flaws of smart city information systems, it is possible to conclude that blockchain systems are the best option for smart city information systems (Idelberger, 2016). This, however, may not be accurate and relevant in all cases, much less universally.
When traditional information systems are exposed to the public and used by tens of thousands of people in a city, they demand a higher level of security and service availability (Maglio, 2009), (Sun, 2016). Blockchain technologies, on the other hand, provide an entirely new mix of technologies to address security and reliability challenges, as well as service availability and fault tolerance (Cai, 2016), (Garay, 2015).
Lombardi (Lombardi, 2012) made a list of smart city components: 1. Smart economy -viewed as the industry or as an aspect of urban life; 2. Smart citizens -knowledge and human capital; 3. Smart governance -e-Government and open data; 4. Smart mobility -logistics and infrastructures; 5. Smart environment -sustainability; 6. Smart living -liveability, quality of life, and security.
A literature review paper (Ćirić, Blockchain Technology Application Areas in the Smart City Information Systems, 2019) maps potential blockchain uses within components of the smart city from (Lombardi, 2012) list. Blockchain-based systems provide strong security combined with total anonymity (Zyskind, 2015), (Xu, 2016); their distributed nature also increases availability; and finally, their nature is interoperable, improving privacy, security, availability, and heterogeneous architectures. "Blockchain as an emerging technology has many good features, such as trustfree, transparency, pseudonymity, democracy, automation, decentralization, and security. These features of blockchain are helpful to improve smart city services and promote the development of smart cities" (Xie, 2019).
Because of these advantages of blockchain technology, there are more possible uses and, as a result, real-world implementations, scholarly discussions, and studies in the areas of the smart economy (Beck et al., 2016), smart transportation (Podovac, 2021), smart tourism (Tomić, 2020), and supply chain management, as well as e-Government (Böhme, 2015), (Khanna, 2021). The main contribution of blockchain to the smart environment is perceived as an impact on the sustainability of software applications within smart cities, supply chain optimization, and new business models in energy systems (Swan, 2015), (Tapscott, 2016), (Zhao, 2016).
There are clear signs that blockchain technology enhances existing systems based on automatic data processing while assuring enhanced security, transparency, and simpler participation of many actors (individuals or companies alike) (Ćirić, Identification of critical success factors for the implementation of the blockchain projects in the smart cities, 2019). Although the term "Information Systems in Smart Cities" is a broad term, this study focuses on software solutions in smart cities that are related to e-government and blockchain technology's areas of application: transactions, payment and exchange infrastructure, smart contracts, identification, confidential data, data storage, voting, and fundraising (Ćirić, Implementation of Blockchain Technology in the Smart City, 2020).
As revealed in the literature review (Ćirić, Implementation of Blockchain Technology in the Smart City, 2020), papers addressing the application areas of blockchain to promote the sustainability of smart cities exist, but they do not contain a comprehensive framework for the implementation of blockchain technologies in the smart city. The existing published research, on the other hand, examines these applications through the fragmented lens of smaller application areas rather than smart city components (Wang, 2016), (Bhushan et al., 2020). This may be the reason why the authors were unable to locate any functioning framework for the implementation of information systems projects (IS) based on these technologies.
According to the authors of this paper, it is vital to identify the areas where this technology may be used in smart cities and to continue to investigate new applications to define success factors and outcome indicators for the implementation of IS projects based on blockchain technology. Applications of blockchain technology in smart cities should be classified into real-life use cases found within the existing smart cities and those found within the scientific literature to undertake more research on this issue. Further research into effectively implemented blockchain information systems is needed to discover smart city components and specific subsystems where blockchain technologies outperformed conventional information systems in terms of performance and availability.
A synthesis is sought through a brief discussion of the most common challenges and limitations of conventional information systems within a smart city and a brief review of the strong features of blockchain technology.
Studies of blockchain technology exist in each given smart city challenge, along with publications on distributed applications (for instance, multilevel authorization (Cordeschi, 2015), and energyefficient resource planning in distributed applications (Yli-Huumo, 2016), (Efanov, 2018). A thorough evaluation and acceptance of practical solutions will hasten the resolution of blockchain technology's current issues and limits (Burgess, 2015), (Yang, 2018) To attain a greater level of sustainability in smart cities, subsystems whose business could be improved utilizing blockchain technology must be identified. Evaluating the challenges and past results of blockchain technology usage will identify the smart cities sub-systems that gain the most from the deployment of this sort of project with the highest project performance.
The information gathered through this scientific method would then be used to identify important success factors and indicators for this type of project execution. (Đurkin, 2018).
A functional framework for the implementation of information systems projects based on blockchain technologies in smart cities can be built based on the identification of the field of effective application of information systems in smart cities, crucial success factors, and outcome indicators (performance indicators). The research and its results presented in this paper are part of the research on performance indicators as a subcomponent of broader research on the creation of a framework for the implementation of the smart city.

Discussion on the performance indicators
As stated in the introduction the research on performance indicators is a part of research on the implementation of blockchain technologies in smart cities.
During the research preparation, the authors conducted a literature review to detect potential performance indicators and use them as starting point for an Expert panel discussion on performance indicators (Keil, 2013). The following list of performance indicators was created based on the author's research in the scientific literature: 1. public interest in the project, 2. governance and leadership support, 3. budget use, 4. time management (time use), 5. project delivery according to the delivery plan, 6. level of system complexity, 7. number of working nodes in the blockchain, 8. degree in technology innovation, 9. end-user satisfaction.
After the questionnaire was checked, in terms of spotting some irregularities, such as the filling out of the questionnaire, the classification of new factors for successful implementation of blockchain technologies was performed, as well as the unification of terminology for the proposed factors, after which the consolidated list of factors was sent to the panelists. Therefore, in the next phase, participants were sent a list of consolidated factors, collected in the first phase of the questionnaire, and grouped into categories and copies of their first-phase responses, where they were asked to confirm that the answers were interpreted correctly and placed in the appropriate categories.
Following Schmidt's (Schmidt, 1997) procedure of ranking type Delphi research, which involves, first, proposing and validating the proposed factors, then narrowing the list to the most important factors and, ultimately, ranking the list of the most important factors, questionnaires were distributed online, through the Google Forms platform. According to this procedure, panelists were asked to rank the proposed application fields, critical performance factors, and performance indicators of blockchain technology implementation based on the five-point Likert scale (Likert, 1932). Before filling out the questionnaire with the participants, all language terminological disagreements that could be subjectively found in questionnaires were clarified. So before filling out the questionnaire, each participant understood very clearly and unequivocally every position they were to evaluate.
The expert panel discussed these proposed performance indicators and several experts proposed additional performance indicators that in their opinion should be taken into the consideration. The following list of additional performance indicators proposed by the experts participating in the research: 1. user base growth over time, 2. environmental sustainability, 3. risk density, 4. policy revision based on the implementation of new data, 5. ease of access, 6. data integrity, 7. resiliency, 8. the number of transactions executed.
Out of all newly proposed performance indicators on the performance indicator "User base growth over/in time" was accepted as a performance indicator and it was analyzed and used in further research.
The authors of this paper would also like to state a different viewpoint on this subject. This can be done by asking the research question such as "Do blockchain projects have different project performance indicators in comparison to nonblockchain IT projects?" or "What are performance indicators specific to blockchain projects?" If the presented lists are assessed from this point of view, only indicator no. 7 from the first list and indicator no. 8 from the second list can be exclusively tied to blockchain technology use. All the other performance indicators can be applied to assess any other IT project. The strength and validity of these performance indicators can vary depending on the IT project type; however, they can all be used to assess any IT project's performance.

Delphi method
The Delphi study does not depend on a representative statistical sample, but rather on a group decision-making mechanism, which requires qualified experts who understand the problem that is being investigated (Paré, 2013). The researchers designed two questionnaires: the initial one and a second questionnaire, asking respondents to revise their original answers and/or answer other questions based on the feedback from the group in the first study. The researchers repeat this process until respondents reach a satisfactory degree of consensus. During the whole process, respondents are anonymous to each other (though not to the researcher) (Okoli, 2004).
For research purposes, experts are divided into panels. A total number of 33 experts participated in the research.
The experts were divided into two groups: 1. Academics (scientists) dealing with blockchain technologies 2. Experts -practitioners dealing with blockchain technologies All the experts had years of experience in the field of ICT and immediate knowledge of blockchain solution implementation projects in smart cities or other ICT projects in smart cities that may be relevant.

Experts
The sample included 33 experts in project management, IT, system development, smart cities, blockchain, or any other relevant areas. Most respondents had work experience, in the relevant areas, from 11 to 20 years (39.39%), slightly fewer respondents had 21-30 years of experience (33.33%), and 24.24% of respondents had up to 10 years of work experience, while the least respondents (3.03%) had experience for more than 31 years (Figure 1).

Figure 1
Expert's work experience in relevant areas: project management, IT, system development, smart cities, blockchain, or any other (in years). Source: the authors.
Most respondents participated in blockchain projects up to 5 times (72.73%), 15.15% of respondents participated in 6-10, while 12.12% of respondents participated in more than 11 blockchain projects (Figure 2). Based on the survey results, most respondents participated in smart city projects, which included new IT solutions up to five times (84.85%). Significantly fewer respondents (9.09%) participated in 6-10, while the least respondents (6.06%) participated in more than 11 projects O N L I N E F I R S T STRATEGIC MANAGEMENT, Vol. xx (20xx), No. x, pp. 0xx-0xx ( Figure 3). Also, according to the survey results, one respondent has so far not participated in the design and implementation projects of information systems based on blockchain in smart cities, one respondent on two and one respondent participated in 10, while the other respondents had one participation in the projects.

Findings
After all rounds of the Delphi method were finished the experts concluded the list of the performance indicators for the implementation of blockchain projects in the smart city.
As the most important specific activities and performance indicators of blockchain technologies, panelists evaluated the following: 1. end-user satisfaction, 2. user base growth over time, 3. use of the budget, 4. time management.

Discussion
According to the author's best knowledge, there is no similar attempt to use the Delphi method to assess these research areas. This study and the discussion on performance indicators as an attempt to provide scientific knowledge that can aid the implementation of blockchain projects in general and in the smart cities as an area of application that directly benefits all of us. As shown in the "Discussion on the performance indicators" chapter experts that participated in the panel proposed 8 additional performance indicators on top of the 9 performance indicators list created by the authors. However, during the process of reaching consensus according to the Delphi methodology, only one of 8 additional performance indicators had consensus to reach 2 nd round of research and have its importance measured. These lists as well as research results are open for further discussion and the authors hope that they contributed to the knowledge on his subject. Since 7 of the performance indicators proposed by the individual experts did not reach a consensus needed to be discussed in the experts' panel, they won't be a subject of further study. It is also important to point out that the only performance indicator that was proposed and accepted by the panel -"User base growth over time" got ranked as 2 nd most important performance indicator for blockchain project implementation in smart cities. The user base or the number of participants in the network can be used to estimate the value of the network according to Metcalfe's Law but the authors were unable to find any mention of "user base growth over time" as a blockchain project implementation performance indicator in the scientific literature. According to the authors' best knowledge, there is no other ranking of these performance indicators in the scientific literature. Further study should lead to additional data and interconnections between all 3 parts of the research: areas of application, critical success factors, and performance indicators. The authors desire to create a comprehensive scientific study that can offer help to everyone trying to implement blockchain solutions in the smart city.