Design and evaluation of a scalable smart city software platform with large-scale simulations

https://doi.org/10.1016/j.future.2018.10.026Get rights and content

Highlights

  • InterSCity platform microservices architecture provides elasticity and scalability.

  • Simulation-based method for realistic smart city workload generation.

  • Extensive analysis of the InterSCity architectural design points out its scalability.

  • Experimental results demonstrate the high scalability of the InterSCity platform.

Abstract

Smart Cities combine advances in Internet of Things, Big Data, Social Networks, and Cloud Computing technologies with the demand for cyber–physical applications in areas of public interest, such as Health, Public Safety, and Mobility. The end goal is to leverage the use of city resources to improve the quality of life of its citizens. Achieving this goal, however, requires advanced support for the development and operation of applications in a complex and dynamic environment. Middleware platforms can provide an integrated infrastructure that enables solutions for smart cities by combining heterogeneous city devices and providing unified, high-level facilities for the development of applications and services. Although several smart city platforms have been proposed in the literature, there are still open research and development challenges related to their scalability, maintainability, interoperability, and reuse in the context of different cities, to name a few. Moreover, available platforms lack extensive scientific validation, which hinders a comparative analysis of their applicability. Aiming to close this gap, we propose InterSCity, a microservices-based, open-source, smart city platform that enables the collaborative development of large-scale systems, applications, and services for the cities of the future, contributing to turn them into truly smart cyber–physical environments. In this paper, we present the architecture of the InterSCity platform, followed by a comprehensive set of experiments that evaluate its scalability. The experiments were conducted using a smart city simulator to generate realistic workloads used to assess the platform in extreme conditions. The experimental results demonstrate that the platform can scale horizontally to handle the highly dynamic demands of a large smart city while maintaining low response times. The experiments also show the effectiveness of the technique used to generate synthetic workloads.

Introduction

Most large cities around the world are challenged by problems related to population growth, shortage of resources, air pollution, traffic congestion, and public safety, among many other modern urban problems. Research on Smart Cities seeks to mitigate these problems using Information and Communication Technology (ICT) to leverage the city infrastructure and resources as part of a highly integrated, diverse, and large-scale smart cyber–physical environment. The goal is to enable systems and applications that will ultimately turn the city into a smart environment, contributing to improve the quality of life of its citizens.

A number of industry initiatives have been successfully adopted in cities around the world, tackling different urban problems. They combine cyber–physical systems with mobile applications to enable the realization of bike sharing services, bus tracking systems, and smart traffic lights, which are now widespread in large urban centers. Smart navigation systems such as Waze and Google Maps are examples of smart city applications capable of serving millions of users by combining geolocation, crowd-sensing, and real-time data. The early adoption of ICT in the context of urban problems has significantly impacted multiple application domains. Although several of the mentioned systems handle scalable data processing and services, they do not offer an integrated infrastructure to facilitate the development of other smart city applications through facilities that can be shared across multiple domains and services.

In the academia, several efforts have been devoted to the study and development of the smart city paradigm, leading to a proliferation of initiatives around the world, targeting different domains such as urban mobility, energy management, and healthcare. Despite their early success, there is a need for more effective solutions for data integration and sharing as several of those initiatives have been developed using ad-hoc approaches [1]. They neither follow a common set of practices and standards nor consider the need for data and resource sharing among the different systems in the city [2], [3]. This hinders the development of smart city solutions that are both sustainable in the long term and reusable in cities with different characteristics, thus creating market islands and raising extensibility, adaptability and interoperability issues.

The use of smart city platforms facilitates the fast development, deployment, and operation of integrated, high-quality smart-city services and applications [3]. Such platforms typically implement a set of common functional requirements in the form of reusable services that application developers can use, including services for the integration of Internet of Things (IoT) devices, as well as for data storage and processing, and for context-awareness. Similarly, the architectures of those platforms may meet multiple non-functional requirements such as adaptability, privacy, interoperability, and evolvability, according to the specificities of the problem that they intend to solve. In particular, in this paper, we are mainly interested in designing and evaluating such platforms for scalability, a non-functional requirement for future cyber–physical systems that is specially critical in the context of smart urban spaces, which may encompass entire cities, with millions of sensors, actuators and humans in the loop [3], [4].

A smart city platform must handle a large number of devices that are part of or are integrated with the city infrastructure. It needs to store and process large volumes of data related to the city, which are continuously produced and consumed by devices and client applications. At the same time, the platform must be able to support thousands of requests from users and from services that run on top of it. The scalability demands thus vary according to the characteristics of the city, as well as those of the deployed applications and services. For instance, a city may start with a pilot project in one of its districts and then expand to other parts of the city as the required infrastructure becomes available.

Despite its importance, platform scalability has not yet received the necessary attention in smart city research. According to a previous systematic study of the literature [3], most of the works that supposedly meet scalability requirements only present superficial discussions of design and implementation decisions that can lead to a scalable architecture. Moreover, they often do not provide scientific evidence of the feasibility of their approaches. Sanchez et al. [5] highlighted that several IoT projects were able to present concrete solutions through new technologies and architectural models, but failed to present conclusive validation of the proposed solutions, which are often restricted to proofs-of-concept. However, demonstrating the actual scalability of smart city platforms presents significant challenges due to the lack of available infrastructure for real experimental setups, as well as the lack of comprehensive datasets.

This state of affairs derives from the difficulty in assessing platform characteristics in real-world scenarios. This difficulty stems both from the significant deployment and usage hurdles in most platforms and from the lack of adequate evaluation techniques.

Middleware platforms for smart cities usually implement complex distributed architectures, requiring considerable effort in documentation and automation to run, configure, and test them. Also, these platforms are often implemented as non-open-source software, hampering their use and study by third parties, as well as the reproducibility of results and collaborative development. Consequently, few projects are truly deployed in production environments, as most of the existing infrastructures are experimental and small in scale.

Even if this was not the case, the research community still needs to advance in the direction of more sophisticated methods, tools, and benchmark strategies to enable a comprehensive evaluation of smart city platforms. One challenging related problem is the generation of representative workloads to assess the performance and scalability of smart city software platforms. Two distinct fundamental actors must be accounted for: (1) client applications, which generate requests using the platform facilities, and (2) the underlying IoT devices, which continuously produce sensed data and/or receive actuation commands. In this context, the use of randomly generated synthetic workloads or a failure to consider either of these external actors can result in experiments that, although useful for evaluating the system in a limited context, do not represent realistic scenarios. This has been observed in most of the reported evaluations of smart city platforms, as exemplified by [6], [7], [8].

In a real smart city, citizens dynamically interact with the cyber–physical infrastructure, triggering multiple events and changing the context observed by pervasive devices and systems. Similarly, platform clients, such as end-user applications and city management services, may vary their interaction behavior with the platform due to events observed in the physical world and real-time data provided by the city. This dynamic behavior of citizens and platform clients needs to be captured and modeled to generate meaningful and representative synthetic workloads.

Available benchmark tools focus on generating workloads for Smart Cities or IoT platforms by extrapolating real sensor traces from various contexts or by emulating the behavior of users based on Web traces. The use of smart city simulators can reduce the problems related to workload generation by allowing large-scale experiments on smart city platforms based on more realistic scenarios. Such simulators implement models that reproduce smart city environments based on the behavior of citizens, IoT devices, and other aspects of the city, such as vehicles, buildings, and lamp posts [9]. Although data produced by a simulator is still synthetic, it reflects the individual behavior of involved actors and their interactions, emulating the dynamic behavior of a city and adapting the simulation accordingly. However, making a simulator interact with a smart city platform is not a trivial task. Notably, the scalability of the simulator itself needs to be addressed, as well as its integration with the target platform.

In 2016, we started the development of InterSCity,1 an open-source, microservices-based platform that provides a cloud-native software infrastructure to support the development of smart city projects, applications, and services. InterSCity has a microservices architecture that contributes to its scalability. Its modularization through small, single-purpose and loosely-coupled interconnected services enables the evolution of its design, which is a valuable property to adequately meet the continuously changing demands of novel smart city contexts and applications.

In this paper, we investigate the scalability of the InterSCity platform using InterSCSimulator, a scalable smart city simulator [9]. In previous work [6], we introduced the InterSCity platform and provided preliminary experimental results indicating the feasibility of our approach towards achieving a scalable architecture. Although we have maintained the main InterSCity modules in relation to our previous work, we have refined important architectural decisions regarding microservices communication and deployment, enabling greater scalability and the integration with the simulator. Also, we have implemented several improvements in the microservices code base related to database queries, caching mechanism, and data serialization to avoid performance issues. This paper brings three new fundamental contributions:

  • A detailed exploration of the current architecture and implementation of the InterSCity platform, supported by a deeper discussion of the design decisions made to meet scalability requirements and more comprehensive experiments considering realistic scenarios and covering broader requirements than the ones of the previous version presented in [6].

  • Advances on performance evaluation of smart city platforms, with the proposal of a new method for workload generation that addresses the dynamism of smart cities by using large-scale simulations integrated with the platform.

  • An application of the simulation-based experimental strategy to evaluate the InterSCity platform based on a Smart Parking application scenario. We provide an extensive analysis of the platform’s scalability properties, advancing the knowledge on the use of the microservices architectural pattern to develop smart city solutions.

This paper is organized as follows. Section 2 discusses related work targeting either architectural proposals for scalable smart city platforms or the generation of workloads for scalability and performance evaluation of smart city platforms. Section 3 explores the design and implementation details of the InterSCity microservices architecture related to scalability. Section 4 presents a brief overview of InterSCSimulator, its integration with the InterSCity platform, and the modeled smart city scenario. Section 5 describes the experimental methodology to assess the platform using the modeled scenario and a comprehensive scalability analysis. Finally, Section 6 presents our conclusions and discusses future work.

Section snippets

Related work

Research on smart city platforms comprises a wide range of areas and related problems. However, in the context of this work, we are mainly concerned with the scalability of these platforms. Therefore, we divide relevant related work into two categories: (1) projects that propose novel smart city platforms that address scalability challenges, and (2) tools and methodologies to generate workloads for experiments on smart city platforms.

The InterSCity platform

Future smart cities will demand high-quality research in multiple areas. The InterSCity project [27] is a multidisciplinary consortium that aims to develop scientific and technological research to address key challenges related to the software infrastructure of smart cities, focusing on Networking and High-Performance Distributed Computing, Software Engineering, Data Analysis, and Mathematical Modeling. The project aims at enabling the development of reusable open-source technologies and

Smart city simulation for workload generation

This section discusses the fundamental steps to enable simulation-based workload generation. This approach is meant to support comprehensive performance and scalability experiments on top of the InterSCity platform, especially considering realistic, large-scale smart city scenarios. In particular, we present InterSCSimulator, a smart city simulator for the behavior of city actors and their interactions in large-scale settings. We also describe the simulated scenario and how we implemented the

Experimental evaluation and analysis

Using the integrated environment described in Section 4, we performed a comprehensive experiment to assess the scalability properties of the InterSCity platform. The experiment consisted of: (I) running a production-like instance of the platform in a cloud environment; (II) enabling an auto-scaling mechanism for the platform’s microservices based on the variation of the workload; (III) setting up the simulator in an isolated environment; (IV) starting the simulation of the Smart Parking

Conclusions and future work

Appropriate software platforms are critical to the feasibility and widespread adoption of complex smart cyber–physical environments, especially future smart cities, requiring considerable development and research effort to ensure their technical quality. Despite the existence of numerous smart city projects, developing architectures to support city-scale environments and evaluating them is still challenging. Thus, this paper brought a comprehensive discussion on the InterSCity platform

Arthur Del Esposte is a Computer Science Master student at the Institute of Mathematics and Statistics (IME) at the University of São Paulo (USP) and has a bachelor degree in Software Engineering at the University of Brasilia (UnB). He participates in research and software development projects, specially within the contexts of distributed systems, microservices architecture, smart cities, FLOSS, agile teams, and other aspects of software engineering.

References (36)

  • MorabitoR. et al.

    LEGIoT: a lightweight edge gateway for the internet of things

    Future Gener. Comput. Syst.

    (2018)
  • SanchezL. et al.

    SmartSantander: IoT experimentation over a smart city testbed

    Comput. Netw.

    (2014)
  • TchernykhA. et al.

    Towards understanding uncertainty in cloud computing resource provisioning

    Procedia Comput. Sci.

    (2015)
  • PCAST

    Technology and the future of cities, report to the president

  • F.J. Villanueva, M.J. Santofimia, D. Villa, J. Barba, J.C. Lpez, Civitas: the smart city middleware, from sensors to...
  • SantanaE.F.Z. et al.

    Software platforms for smart cities: concepts, requirements, challenges, and a unified reference architecture

    ACM Comput. Surv.

    (2017)
  • L. Sanchez, J.A. Galache, V. Gutierrez, J.M. Hernandez, J. Bernat, A. Gluhak, T. Garcia, SmartSantander: the meeting...
  • A.M.D. Esposte, F. Kon, F.M. Costa, N. Lago, InterSCity: a scalable microservice-based open source platform for smart...
  • PereiraC. et al.

    Benchmarking pub/sub IoT middleware platforms for smart services

    J. Reliab. Intell. Environ.

    (2018)
  • D. Preuveneers, Y. Berbers, Samurai: a streaming multi-tenant context-management architecture for intelligent and...
  • E.F.Z. Santana, N. Lago, F. Kon, D.S. Milojicic, InterSCSimulator: large-scale traffic simulation in smart cities using...
  • ChengB. et al.

    Building a big data platform for smart cities: experience and lessons from Santander

  • YamamotoS. et al.

    Using materialized view as a service of scallop4sc for smart city application services

  • GirtelschmidS. et al.

    Big data in large scale intelligent smart city installations

  • KrylovskiyA. et al.

    Designing a smart city internet of things platform with microservice architecture

  • PetroloR. et al.

    Towards a cloud of things smart city

    IEEE COMSOC MMTC E-Letter

    (2014)
  • PereiraR.L. et al.

    CitySDK tourism API-building value around open data

    J. Internet Serv. Appl.

    (2015)
  • PuiuD. et al.

    Citypulse: large scale data analytics framework for smart cities

    IEEE Access

    (2016)
  • Cited by (60)

    • Microservice based scalable IoT architecture for device interoperability

      2023, Computer Standards and Interfaces
      Citation Excerpt :

      The “InterSCity”, an open-source platform based on microservice architecture, is suggested for allowing rapid development of various services, systems, and applications required for the smart IoT. The scalability can be achieved in this platform due to the modular design and technology, independent deployment, and functional decomposition [12]. Many context-aware IoT applications are also examined in a smart environment to determine their capabilities in terms of real-time decision-making and scalability [31].

    • Big data architecture for connected vehicles: Feedback and application examples from an automotive group

      2022, Future Generation Computer Systems
      Citation Excerpt :

      In the literature, we also find numerous big data solutions (platforms and architectures) related to smart cities and other specific domains. Among these solutions, we mention: PROMENADE [31](Graph-based general-purpose big data platform for smart-cities), DIMMER [32] (City energy efficiency), InterCity [33] (Parking application), Snap4City [34] (Mobility and transport), Sipresk [35] (ITS and traffic congestion), BRT [36] (Transportation planning), CEFIoT [37] (Surveillance camera system), NIMBLE [38] (Industry 4.0), Druid [39] (Exploring log data), RADStack [40] (Online advertising, based on [39]), SmartSantander [41] (Monitoring and irrigation), CiDAP [42] (Monitoring, based on [41]), Scallop4SC [43] (Smart houses). To conclude this section, we state that the big data architecture of Groupe PSA was initially and mainly inspired by the Lambda Architecture.

    • Big data analysis and optimization and platform components

      2022, Journal of King Saud University - Science
      Citation Excerpt :

      It can greatly enhance the storage capacity and utilization rate of traditional database. At the same time, the emergence of CloudDB technology optimizes many back-end functions of traditional databases, with the advantages of high personalization, scalability, and support for effective distribution of resources (Esposte et al., 2019). So this side is mainly for the first two to introduce.

    View all citing articles on Scopus

    Arthur Del Esposte is a Computer Science Master student at the Institute of Mathematics and Statistics (IME) at the University of São Paulo (USP) and has a bachelor degree in Software Engineering at the University of Brasilia (UnB). He participates in research and software development projects, specially within the contexts of distributed systems, microservices architecture, smart cities, FLOSS, agile teams, and other aspects of software engineering.

    Eduardo Felipe Zambom Santana concluded his MSc (2010) and BSc (2007) at Federal University of São Carlos and currently is PhD student at University of São Paulo and lecturer at Anhembi Morumbi University. His main research topics are Smart Cities, Software Platforms, Cities Simulation, and Distributed Systems.

    Lucas Kanashiro is a Computer Science Master student from the Institute of Mathematics and Statistics (IME) at the University of São Paulo (USP), working on behalf of the Free/Libre/Open Source Software Competence Center (CCSL). He received his Bachelor’s in 2015 from the University of Brasília (UnB) where he majored in Software Engineering. His research interest area are: Free/Libre/Open Source Software development, Source code metrics, Smart Cities, Distributed systems. Moreover, he is a Debian developer and a Free Software enthusiast.

    Fábio M. Costa is a full professor at the Institute of Informatics at the Federal University of Goiás, Brazil. He received his Ph.D. in Computing from Lancaster University (UK) in 2001. His research interests are in the area of adaptive distributed systems in general and on reflective middleware in particular, adopting concepts from model-driven engineering, notably , in order to make reflection more tractable and predictable. Results of this research have been applied in areas such as cloud computing, smart grid, communication platforms, and ubiquitous computing.

    Kelly Rosa Braghetto is an Assistant Professor at the Department of Computer Science - Institute of Mathematics and Statics - University of São Paulo since 2012. She has a MSc and a PhD in Computer Science from University of São Paulo. Her main research interests are: Databases, Workflow Management, Distributed Systems, Performance Evaluation of Computational Systems, and Formal Methods and Models for System Design and Analysis. She has participated in several research projects, developing new solutions for process and data management.

    Nelson Lago concluded his MSc at the Institute of Mathematics and Statistics (IME) - University of São Paulo (USP). Currently, he is the technical manager at the Free/Libre/Open Source Software Competence Center (CCSL). His main research interests are Free/Open Source Software, Distributed Systems, and Music Computing

    Fabio Kon is a Full Professor of Computer Science at the University of São Paulo and a Visiting Professor at the MIT Senseable City Lab. His research interests include Smart Cities, Big Data Processing, Data Science, Distributed Systems, and Startup Ecosystems. Fabio has been in the Scientific Program Committees of over a dozen high-impact international conferences in the past two decades and is currently the Editor-in-Chief of the SpringerOpen Journal of Internet Services and Applications. Fabio is an ACM Distinguished Scientist and a Special Advisor to the Scientific Director for Innovation programs at the São Paulo Research Agency.

    This research is part of the INCT of the Future Internet for Smart Cities funded by CNPq, Brazil, proc. 465446/2014-0, CAPES, Brazil, proc. 88887.136422/2017-00, and FAPESP, Brazil , proc. 2014/50937-1 and 15/24485-9.

    View full text