COVID bell – A smart doorbell solution for prevention of COVID-19

2.1 Smart doorbells

Smart doorbells have become an essential and obvious part of the fully integrated implementation of smart homes. These smart doorbells help to increase the efficiency of security by avoiding unwanted access such as robbery and invasion as well as to keep a track of people entering the smart home [2,3,4,5]. The controller of these doorbells can easily answer the bell and decide whether to let the guest enter or not accordingly through adaptive learning and various other technologies [6]. As smart doorbells have been extensively used, newer and upcoming technologies have been introduced to improve the existing systems [5,7,8,9]. The issue with the current design lies in its complexity. This means that the working mechanism or the implementation of these integrated solutions is usually vague to the eye of the user. In case of failures or breakdowns, the user will need to contact a professional for repairs and maintenance. It is highly likely that the whole system may have to be replaced because of the failure of a component in the system, and such systems are not fully feasible for real-life implementation.

2.2 Dashbell

This smart doorbell system integrated with the already implied smart home systems goes a notch above the regular smart doorbells. The system uses a simple design based on Raspberry Pi, AWS (Amazon Web Services), and an Android app. The system like any smart doorbell sends a notification to the user’s mobile through the application in which AWS serves as the link. The user can see who the visitor is and then decide whether to allow entry or not. What sets the dashbell apart is that it is more cost-efficient than the existing smart doorbells. It uses an Amazon Dash Button which is easily available in the market for 4.99 US Dollars or 390 Indian Rupees [10,11]. Also, as the system design is simple and easy to understand, anyone can understand the design and resolve the problem in case of a breakdown or malfunction. Further individual parts can be easily replaced instead of replacing the system as a whole. The issue with Dashbell is that even though it reduces the overall cost, it is still a bit costly to be used by the mass. This indeed can be made cheaper without affecting efficiency and functions. Another downside of the proposed implementation is that it does not provide a contact-free experience which may lead to contact spread of COVID-19 and other viruses and does not gather information about the guest’s temperature and whether they are wearing a face mask or not. Also, the solution is only implemented for Android users and users with other operating software like iOS [12].

2.3 Survey on Internet of Things (IoT) and AI-based smart doorbell system

This survey was done to analyse the existing software technologies and suggest better alternatives for fast and accurate face detection. This survey points out the complexity of existing haar models to detect the face of a person. Haar models not only take time but also add extra steps to the procedure of face detection [13,14]. The major advantage of the model is that it is serverless, which makes it easier to add or delete functions as the pipeline grows. This article [15] proposes the use of Kairos face recognition, which is a service provided by Kairos company. This model does offer accuracy and speed but it cannot detect face masks, which is our added functionality to the smart doorbell system. Moreover, the Kairos software is a paid software that will cost more and more as the number of doorbell devices increases. One of the biggest disadvantages of this model is that it starts detecting the face when the doorbell is pressed [14]. This is not safe as COVID is a surface-spread virus. We capture the image as soon as the person hovers their hand in front of the doorbell. We have carefully done the literature survey to find the best possible open-source free-of-cost face mask detection model that proves to be highly accurate and efficient. We have chosen a very lightweight model that integrates easily with the app. The neural layers of our model are arranged in such a way that increases the overall inference speed of face mask detection. Our model performs considerably well in dimly lit areas. It is also able to identify a face in case of backgrounds with high disturbance.

2.4 Smart doorbell design using federated deep learning

This article demonstrates a Doorbell Security System, which uses Federated Deep Learning to identify objects and people near the door and deploy it on a scalable serverless architecture [16]. Federated learning is a new and upcoming field in the world of Machine Learning. It works by the means of Model Aggregation over Data Aggregation providing faster and more accurate results as compared to other methods [17,18]. The article demonstrates an architecture consisting of a video doorbell that will stream video to the Federated Server and Cloud [19]. The Federated server will use the provided data to extract features and provide the user with valuable information in an iOS Application. It can detect people, objects, cars, animals, and more [17,20,22]. It also has a notification system to notify users whenever someone or something is detected by the doorbell. The article brings up an important issue which is the privacy of user data and aims to deploy Homomorphic Encryption and compute on Secure Multiparty Computation, which enables multiple parties to collaboratively work on an agreed-upon computation, thus preventing data leaks [21]. This type of model can incorporate the detection of Masks to identify people complying with the COVID protocols [23]. The current proposed model in the mentioned paper seems to be using a RESTFUL approach towards the communication between hardware and iOS application. It alerts users via Push Notification Services. This can be made more efficient with a real-time approach using a Web-socket implementation. This implementation combined with temperature detection, Mask detection, and the ability to remotely control the door would result in an almost perfect system that can help us curb the spread of viruses.

2.5 Development of COVID-19 detection smart doorbell based on IoT

As we know, Necessity is the mother of all innovations, and this system is born out of the need of monitoring visitors’ temperature and reducing contact with possibly infected people as much as possible. The system proposes an Infrared system be installed on the bell switch, which senses the person’s temperature upon contact using their fingers or thumb. The recorded temperature is displayed on an LCD screen for the residents to monitor. The temperature is also cross-referenced with standard fever temperatures to notify the user about the risks involved [24]. The proposed solution offers a good window into the development and integration of technology, mainly IoT in today’s day and age where security risks are not necessarily limited to unwanted break-ins and uninvited guests. It treats and justifies COVID-19 spread as a security risk and rightfully so. The system comprises a Raspberry-pi along with a Fingerprint sensor, Infrared sensors, and a camera. It adds a layer of infrared temperature scanning to existing solutions as a means to detect COVID-19 using visitors’ temperature readings. The major drawback of the proposed system is that it does not necessarily minimise contact as the visitor has to touch the doorbell for their temperature to be recorded. If an infected person touches the doorbell, it leads to a high risk of contact spread to the next visitors. It also does not take into account asymptomatic carriers of the virus as temperature for them remains within normal range. Although there is not any existing solution to detect the asymptomatic carriers apart from a sure-shot RTPCR test, the spread can still be minimised by following proper guidelines such as wearing masks. The system also does not offer a remote solution where the resident need not necessarily go to the door to allow a visitor to enter back in. A good example would be if a house has multiple residents and one of them is at the door. It is usually best to allow them to sanitise properly before coming in contact with other people. A remote solution could allow the outsider to get in without coming in contact with others and sanitise properly as per guidelines.

2.6 Development of COVID-19 detection smart doorbell based on IoT

This article successfully implements a smart doorbell that secures the home and office environment [25]. The major advantages of the proposed model include an HD camera and an intruder alarm system. The system turns on the alarm if anyone tries to steal the installed device itself. The video call and audio features allow the user to communicate with the person standing at the door. It is connected to smart artificial intelligence tools like Alexa and Siri, which enhance the ease of use. They also suggest an automatic power supply system that ensures the device never turns off. This increases the battery life of the system as well. But looking at the current scenario of COVID, we decided to add more relevant features that are more advantageous like the face mask detection and temperature monitoring system. What makes our system more comprehensive is the integration of a mobile app that allows the user to check and control everything from the mobile app itself. Furthermore, we are using a real-time database instead of a primitive SD card to transfer and feed data into the device. Our model smartly switches between WiFi networks and Bluetooth based on the availability of the network and the proximity of the user from the bell. We propose to put the data collected to better use by analysing the temperature data for COVID spread situations in a particular region. The most important aspect of any device is its availability and cost-effectiveness. We have used commonly available hardware modules that ensure the wide availability and low cost of the system.

2.7 JustIoT IoT based on the Firebase real-time database

This article mainly consists of four components: back-end Google Firebase real-time database, front-end SPA (Single Page Application) web monitoring program (including mobile monitoring App), controller software hardware, and intelligence server that supports MQTT connection and condition control. The web portal receives data from microcontrollers like Arduino allowing users to control rules and monitor and control the system. The web portal uses a management-based web page on the Angular front-end, which is connected to the Firebase real-time database. MQTT server is used to establish a connection with Arduino. To further enhance the system in the future, the article states that a machine learning framework can be integrated to make it more intelligent [26]. Even the latest variant of CNN termed Capsule Network analysed and employed in previous studies [27–29] could also be used for increasing the intelligence in the proposed approach.

2.8 Impact of COVID-19 on education

The article puts forward an analysis of responses to a survey in which students of different organisations were asked about their opinions and feedback on the virtual/online classroom and education system. The study states that instead of utilising the benefit of e-teaching during the pandemic teachers rather conducted online classes in urgency and did not focus on building the students’ skills and experiences. In its findings, the article also talks about how teachers and academians alike are not familiar and trained with online teaching platforms and tools, and hence, there is a gap in communication between the mentors and the students. The survey shows how the lack of attention and seriousness affected the education of students [30].

2.9 Role of IoT to avoid spreading of COVID-19

The article presents a solution to prevent the spread of COVID-19 using an IoT system which consists of an infrared (IR) thermometer, an IR sensor, a smartwatch, an optical camera, and an IP camera. The suspected zone is scanned using edge devices (the cameras and the sensors), and as soon as the high temperature is detected, the camera captures the image of the suspected person and sends the image and GPS location of the person to a health officer and action could be taken accordingly. For transferring and storing the data send to the officer, the system uses a secure cloud architecture within which an ML algorithm runs, which tries to identify the person and analyse the risk of spreading based on the data provided.

3.1 Hardware

The hardware system design has been implemented in such a way that it is cheap, easy to comprehend for the user, and at the same time effective in comparison to existing smart doorbells. The system mainly uses an Arduino UNO as the microcontroller and different sensors and actuators like the HC-SR04 ultrasonic sensor, MLX90614 infrared temperature sensor, and various others. These elements were used keeping in mind their compatibility with the system, their price point, and most importantly their efficiency and simplicity. The hardware connects seamlessly to a Cloud Architecture Hosted on Firebase, which in turn communicates with the client-side app, developed on Swift using Xcode for iOS. Each element of the hardware and software side of the system and their respective function is explained and analysed on how to maximise their output, which are discussed in the following sections.

3.1.1 HC-SR04 ultrasonic sensor

This inexpensive sensor (Figure 1) has a range of 2–400 cm of non-contact measurement capability and a ranging accuracy of up to 3 mm. This sensor is widely used in mini-projects and has been proven to be effective in most situations. We are using the ultrasonic sensor to detect the presence of a person at the door as they wave their hand in front of the sensor. This sensor has a simple integration system and is inexpensive. In case of a breakdown, a person can easily buy it as it is widely available in the market, and the repair or replacement is also simple to implement.

Figure 1

Flow chart of the integrated system.

3.2 Software

3.2.2 Swift

Swift is an open-source, multipurpose, and multi-paradigm language based on Objective-C (Figures 1 and 2). It is developed and maintained by Apple Inc. and is also used to develop Native applications on Apple platforms like iOS and macOS. This is our language of choice for developing our client-side application as it provides robust performance leveraging Native APIs to give users a smooth experience. It offers multiple frameworks for developing applications but the one we use is UIKit [18]. It will not only allow us to develop the User Interface but also implement several client-side logics as per requirements.

Figure 2

Mobile application working and interface.

4.1 Hardware

The ultrasonic sensor sends a signal when a person’s hand is found in proximity to it. This triggers the system as a whole, and the Arduino Uno gets activated. As soon as the activation takes place, the MLX90614 sensor notes the temperature of the visitor and sends it to Arduino, along with the ESP32 camera sending the captured image to Arduino. The Arduino collectively sends this information, i.e., the temperature and captured image to Firebase through ESP8266 WiFi Module or HM10 BlueTooth module (as required).

4.2 Software

As the image is captured by (Figure 1), it is uploaded to Firebase Realtime Database as a Base-64 string along with the temperature and time stamp of the captured image. Once the image is uploaded to Firebase RTDB, the client-side library automatically triggers a call to the database for querying data as it works on Web-Socket-based implementation, which is responsible for its real-time behaviour. This not only reduces the number of times an API is called but also allows us to have a working application that reacts in real time. The iOS app will query the data and notify the user about the visitor. The user can then check the vitals of the visitor such as temperature on the app and trigger the opening of the door by sending a confirmation to Firebase RTDB, which is then received by Arduino. Arduino triggers the servo motor to open the Door as per the command received. The vitals are analysed on the app itself, which in return alerts the user whether or not it is safe to let the visitor in. The Vitals are analysed in the following way:

• The temperature is analysed by checking whether the user’s recorded temperature falls in the safe range or not.

• To analyse and detect face masks in the image, the app uses a swift written library that makes use of (AIZOOTech), which is trained with 7971 custom images in addition to the public data set available at WIDER Face and MAFA. The model resizes the image captured from the camera into 260 × 260. It comprises 24 layers in total while the backbone layer only consists of eight layers to make the model light. What makes the model robust is that the convolution layers are merged with the batch normalisation layers; therefore, the output is generated faster.

In order to make the above process more efficient in case of low network connection, a Bluetooth module on the hardware allows forming a connection with the iOS App using BLE. This way all the communication happens over Bluetooth as opposed to via the cloud. The data are captured by the hardware and sent over Bluetooth as encoded data. Upon receiving these data, it is cached by the App and processed as mentioned earlier. The final trigger for opening the door is also sent through this way or path. The cached data can then be saved or stored in a remote database for future work. This way of communication is integrated as a fall-back mechanism when both the modules are close-by and a network connection over the world wide web cannot be established. This makes the system more efficient and reliable for all types of scenarios.