IoT based Smart Retail System with Social Distancing for Covid19 Outbreak

Covid19 pandemic caused by infection with the severe acute respiratory syndrome corona virus2 is continuously spreading all over the world. The impact of covid19 has been fallen on almost all sectors of development including retail which plays a major role in day to day life. In this paper, we proposed an efficient methodology to create a safe environment of people in retail that contributes to public safety. The proposed smart retail system is focused on the real time monitoring of shopping malls which includes grocery, departmental stores, clothing shops, jewellery shops and shops with food essential products. At first a Mobile Application is developed using Android Studio for prebooking of shopping by users/customers. Using Anaconda Navigator a deep learning trained architecture is developed on distinguishing people with and without face mask. A wearable device is developed using ultra wide band radio technology to ensure safe social distancing which would alert customers as soon as the violation of social distancing is detected. The smart retail system also includes IoT based smart shopping cart with RFID sensors for the customers to check for the availability of items in the web server via Nodemcu and for automatic bill payment. The proposed methodology is also suitable for Religious places, Cinema Theatre, Training centres and Browsing centres.


Introduction
The Corona virus Disease 2019  give rise to Severe Acute Respiratory Syndrome corona virus 2 (SARS-CoV-2), has given rise to a global pandemic. People infected by corona virus are transmitters of disease which makes the covid19 count increasing rapidly through out the world. Covid19 has started on 12 th December, 2019 in Wuhan city of China and it was observed that people were infected by novel corona virus from a shop that sales fresh fishes to people. Within a week, the government of China perceived that millions of people got infected with covid19 in China. Furthermore, many countries are infected with corona virus.
SARS-CoV-2 is continued to spread around the world and quickly became the cause of COVID-19 global pandemic in 2020. Corona virus changes in its genetic structure because of the process known as mutation. One of the most dominant variations of corona virus were called D614G has sparked around the mutation of corona virus.
The affected countries all over the world made a decision to prevent the people from corona virus disease and announced the Lockdown. Due to lockdown, Consumers encircling the world start to shunt human interaction and purveyors are imitative to a reconcile. purveyors recognized that the global response to the novel COVID-19 virus will have a monumental wreck on their business.
Social distancing and wearing masks are verificatory to be an efficacious measure to prevent the dispersal of covid-19. The World Health Organization (WHO) instructs to follow these precautions to be aware of covid-19 spreading. Body torridity measurement is used for screening test for covid19 disease. Each and every public places including cinema theaters, malls, departmental stores where people tend to crowd together, the authorities prioritized to measure body pyrexia of each person to avoid the dispersal of covid19 to other people. While checking the body pyrexia of a person if a person has more than 98.6 degrees, he/she will be addressed to take covid19 test. Where the normal body pyrexia of a person is 98.6 degrees. If the torridity is more than that he/she should definitely take the covid19 screening test until the health record says whether he/she is positive or negative for covid -19.
Mobile phones have become our communication hub in today's world. To reach out our daily accommodation such as food, daily beverages we need to reside out of our home. But due to pandemic situation it is unsafe one to reach out of our home. So we developed a system which allows people to prebook for their shopping in malls, departmental stores and markets. This was applicable through the suppuration in android studio which summed in prebooking for shopping where in shopping mall or departmental stores the purveyor allocated time slot for hours and restricted people count to enter into shopping. This helps to avoid unnecessary crowd gathering in public places.
In some countries, second wave of corona virus disease has started to dispersal which is of same corona virus and mutated corona virus. The second wave affected countries are Russia, United States, United Kingdom, Spain, France, Italy, Brazil, Belgium, south Korea, Iran, Germany, Czech Republic.   figure 1, it is illustrated that the dispersal of corona virus across different countries on dated upto April 5, 2021. This graphical representation demonstrates the infections caused by corona virus to people week by week from January 3 rd week to April 1 st week. The regions mentioned above in the graphical representation were Western Pacific Region, Region of the Americas, Eastern Mediterranean Region, Southeast Asia region, European Region and Africa.
This paper aims at designing a system for IoT based Smart Retail with social distancing for Covid19 outbreak where people were unable to came out during this pandemic situation because they were afraid to go out for shopping even for their daily requirements.
The reminder of this paper is arranged accordingly. The Design of IoT based Smart Retail system is described in section 2. Section 3 analyzes the experimental results obtained from the designed system. Section 4 concludes the work of the proposed system and section 5 demonstrates the future work of this idea and their scopes.

Design of iot based smart retail system
In this paper, we proposed methodology for the design of IoT based Smart Retail System with Social Distancing for Covid19 Outbreak which is divided into four modules.
Initially a Mobile application is developed using Android Studio. Using this application customer can pre-book for shopping time slots and receive the acknowledgment mail. Customer can be asked to confirm their visit during the allocated time slot by sending them a confirmation mail and SMS before two hours of shopping time slot. Once they come for shopping to shopping mall there will be a gate entry check to verify their time slots using RFID tags which will be received by them by entering their registered mobile number and to hunt out whether the customer is wear out a face mask. After the successful verification at gate entry customer will be allowed to purchase items along with a wearable device to monitor safe social distance limit and smart shopping cart attached with RFID reader for automatic bill payment. The process of this experimental design is explained using figure 2.

Module 1-Android mobile application for prebooking for shopping
This module deals with the suppuration of android mobile application for pre-booking for shopping. For suppuration of the mobile application, the platform used is android studio and the language used is Java and HTML.

User end
At the user end, if the user has the account in the mobile application they can login into their account and book for shopping. Otherwise, user has to create a new account by registering in the mobile application and then they can book for shopping. After logging in, the user can see the details of the shopping mall in the menu. The menu consists of options such as Shops and stores, Bookings, Feedback, Rate us and Settings. The shops and stores option contains list of shops with available products. In the bookings option, user can book for shopping with the help of available time slots and need to make online payment for the confirmation of booking.

Retailer end
Retailer can collect and store the database of the users who book with their shopping mall. With that database retailer need to assign RFID tag or card which contains the information of registered mobile number of user where used for the purpose of verification of slot regarding users came for shopping in their booked time slot or not.

Module 2 -Shopping mall gate entry
In this module shopping mall gate entry check will be divided into two phases: (i) Customer access for shopping and (ii) Face mask detection.

Phase 1 -Customer access for shopping
In this phase the RFID reader connected to the Arduino Uno board is used to check slot verification of users. For verification of slot for users, the platform used is Arduino IDE. When the customer is coming for shopping during their allocated time slot, their presence/visit is verified using their registered mobile number and RFID card. They scan their RFID tag using RFID reader. The verification of allocated time slot at gate is done to control corona virus dispersal. After verification of the time slot they will undergo the next phase of face mask detection.

Phase 2 -Face Mask Detection
This phase deals with the face mask detection of users coming for shopping to mall. Face mask detection mechanism can be implemented using OpenCV, Tensorflow/Keras. The platform used for face mask detection is anaconda navigator -jupyter notebook. This mechanism is developed to avoid rapid spreading of Covid-19 from infected people to normal people.
Once the user is detected without a face mask, an alert message is triggered to the user. The block diagram of face mask detection mechanism is depicted in figure 5.

Building the face mask detection
The face mask detection is done by using machine learning algorithm through the image classification method known as MobileNetV2. MobileNetV2 is a classifier based on Convolutional Neural Network algorithm. Here we are using Convolutional Neural Network algorithm in face mask detection.

Convolutional neural network (CNN)
Convolutional Neural Network are a type of deep neural networks that can identify and classify particular features from images. There are 3 types of layers that make up the convolutional neural network algorithm they are i) Convolutional layer ii) Pooling layer iii) Fully connected layer. In addition to these three layers there are two important parameters which are the dropout layer and the activation function. It takes input, assign values to the objects in the image or video which will be useful to differentiate the inputs from another input to give the output.
The CNN learn to optimize kernels. Since they are hand engineered traditionally. The convolutional layers, max pooling layers and fully connected layers are the layers used for the classification. At first features from images are extracted by one or more convolution layers and pooling layers. Then all the image features are mapped to from the last convolution layer and transforms into 1D vector for full connection layer. The output layer classifies the input image.

Convolutional layer
Convolution layer is the pivotal production wedge of convolutional neural network. It is used to convolving one object to another. The main feature of convolutional layer is to unsheathe attributes such as boundary, colors from the input. When convolutional layer starts to track deeper things it also tracks about parts of the face, shapes, digits, letters and words. The process of this layer takes place through by scalar product of two arrays where first array is known as kernel and second array is known as incommodious segment of the array image. With the completion of the convolution process, we have a accent array with clear trait than the actual trait of the image.

Fully connected layer
In this layer we can do our classification process. Up to this layer we have only performed by highlighting configuring features in an image and by reducing the proportions of the image. Now we are going to convert the input image into multi-level fully connected architecture of our suitable form. Then we can demolish the input image towards one column raster. This output is given to a progressing neural network and back promulgation is applied to every redundancy of training. Then the model can classify the images using this layer. Now we can train the model using regression with back promulgation.

MobileNetV2 architecture
MobileNetV2 is based on inverted residual structure where the connections of residual structure are between the connected layers. MobileNetV2 is build upon the ideas from MobileNetV1.There are two types of convolution layers in mobilenetv2 architecture. They are 1 cross1 convolution and 3 cross 3 depth wise convolution. MobileNetV2 uses depth wise separable convolutions, width multiplier, linear bottlenecks and shortcut connections. MobileNetV2 is a light weight image classifier and can be used to perform face mask detection.
We have used MobileNetV2 architecture to recognize the face mask datasets which helps to identify and classify the datasets and differentiate the images into mask and no mask images. With the purpose of recognition we had implemented mobilenetv2 as a light weight image classifier. We have used MobilenetV2 as a foundation one which acts as a network and replace the crust of foundation network with a convolutional layer which we had studied previously in convolutional neural network and a softmax classifier.
Then we applied dropout layer which will be added only when trained data is true and dropout rate of 0.1 is applied to the previously added convolutional 2 dimensional.The Pre-trained MobilenetV2 was used to extricate features of the trained images and the softmax classifier is used to classify the image features. We have trained the face mask detection by using Adam Optimizer of 0.001 learning rate.

Data collection
For building the face mask detection paradigm, we need to collect the dataset of people with mask and without mask. The dataset is trained on data with people who wear mask and who do not wear mask. Dataset is incorporated of 2250 images in which 1315 images with people wear out face masks and the rest 935 images with people who do not fret face masks. The dataset collected is labeled into two types : one is with mask and another one is without mask.

Dataset preprocessing
We need to preprocess the data which are resizing the image, converting the image into the array, preprocessing input using MobileNetV2 architecture on top of Keras layers and labeling the images with frame size. The resizing of image is done using preprocessing method which is difficult one due to the potency of training the model images. Here we are resizing the image into 256*256 pixels.
Then we can convert the image into the array and prepocess the input image using MobileNetV2 architecture and label the images.Here we used Global Average Pooling 2D pooling technique and activation function of ReLU konown as Rectifier Linear Activation function, dropout factor and configure the learning rate of output layer with activation softmax.

Dataset training
We can train the images which we have collected from that one image is selected and trained using OpenCV, Tensorflow , Keras with Tensorflow backend with included python library known as saved model and then serialize the face mask detector to the disk. We use the Adam optimizer with initial βparameter and learning rate of 1 × 10−4. We trained the model image for 10 epochs and dropping learning rate by a factor of 0.01. Once the dataset is trained we can load the mask detector by face detection and test the images via static image or live video.

Implementing the face mask detection model
The face mask detection model is implemented using webcam of a computer which is live video. The video runs from frame to frame, then the convolutional neural network algorithm works using MobileNetV2 classifier. If a face is detected through the frame, proceed to resize the image to pixels and predict the image from the saved model. From that a image is labeled that the person is wear out a mask or not along with a message. If a person wear mask it shows welcome message otherwise it alerts the person to wear a face mask.

Module 3 -Wearable device for social distancemonitoring
In this module social distance can be monitored by wearable device. The platform used for the suppuration of a smart wearable device is Arduino IDE software. During covid19 pandemic situation, people need to keep social distancing of 6 pes or 2 metres from other people in public places and most crowded area. Here we had designed a wearable device which consists of ultrasonic sensor HC-SR04 to alert people whenever they tend to close to other people.
The Ultrasonic sensor is unaware of person, building, object, walls and everything and the sensor hunt outs regardless of color, material and surface. In our proposed system we had implemented the ultrasonic sensor with buzzer and Arduino Uno as a hardware circuit where it buzzers whenever the object or wall or person comes near the person's wearable tag on their front side. When it buzzers people can view by alerting themselves whether a person is nearer to them or any object isplaced. Human brain does not always aware of maintaining social distance so we came up with this device. So we have came up with a solution that is during shopping customers need to wear a wearable device to maintain social distance from other people.The block diagram of wearable device for social distance monitoring is depicted in figure 6. The condition implemented in that wearable device is when the distance between a person and other person is lesser than 6 pes the buzzer gets start blowing louder to alert the customer to maintain social distance from other and on other side if the distance between a person and other person is greater than 6 pes the buzzer will not blow.

Module 4 -Smart shopping cart
In this module, the smart shopping cart consists of nodemcu, RFID reader, RFID tag or card and web server. The platform used for smart shopping cart is arduino IDE software with RFID library package. When the customer chooses their product then they have to swipe the product near to the RFID reader in order to devour the RFID tag affixed to the product. The frequency range of RFID reader is 13.56 MHz which can devour RFID tags at a distance from 1 to 12 inches.
The RFID reader devours the RFID card when it is placed near the RFID reader, it devours the UID (Unique Identifier) information that is present in the RFID card or tag through the ElectroMagnetic waves present on the reader and the micro-chip which is black in colour present in card. RFID reader checks along the information present in the card if it is valid then the details of that RFID card attached to specific product will be displayed on web server affixed to the smart trolley along with the Total bill.
After completion of shopping, the customer has to click the pay now option unveiled on the web server and using their debit/credit card they can pay their bill through online payment gateway which will be accessed by any bank accounts. The block diagram of smart shopping cart is depicted in figure  7.

Module 3 -Wearable device for social distance monitoring
Whenever the similar device of a smart wearable device is encountered less than 2 meter or 6 pes it raises an alarm to a person who is wear out that smart device for maintaining a safe social distance. Figure 11. Smart Shopping Cart using RFID technology and Bill will be displayed on the web server using NodeMcu.

Conclusion
In our paper, a smart retail system for Covid-19 outbreak is proposed based on IoT. The Smart Retail System offers multiple benefits during this pandemic situation such as enabling contactless shopping, maintaining social distance while shopping, more product engagement in shopping malls using smart retail solutions and contactless billing. The major advantage of this project includes safe environment for customers by the detection of face masks and prebooking for shopping in the time slots, the wearable device alerts customer to maintain social distancing while shopping and the smart shopping cart reduces the consumption of time at billing counters. The proposed system increases the overall efficiency and this method can also be applicable in religious places, coaching/training institutes, cinema theatre and educational institutions for maintaining social distancing and the detection of face masks. In future, the proposed system can be improved by the suppuration of voice alert in face mask detection and payment of products can be done automatically from customers bank account while they checkout.

Future work and their scopes
In our present system, we had used ultrasonic sensor with Arduino Uno as a wearable device for social distance monitoring where ultrasonic sensor is unaware of people or object and it will hunt out only at person's front side so people can be aware only on their forward side. In future work we can implement this wearable device by using MaxBotix ultrasonic sensors to hunt out the human presence as a cap of wearable device so it can hunt out on person's four sides. This will helpful to maintain social distance from other people. In face mask detection, we can add up with voice message as an alert message to indicate the person to wear a face mask to avoid the dispersal of Covid19. At present situation every public place authoriser is checking temperature so we had not implemented in our project. Therefore we can add up with the check constraints such as only if face mask wore and pyrexia is normal then only they will be allowed for shopping. This will be done with the help of machine learning algorithm.