Abstract
Food is one of the integral parts of human life making the quality of food one of the prime factors in its selection for consumption. In order to maintain the food quality, it must be taken care of from the very first step where its quality may be affected, that is, warehouses. Food safety and safety of its warehouses is one of the major concerns, because many people lose their lives due to poor food quality. A robot that can ensure the safety of both food and warehouse can be one of the possible solutions, because taking care of huge warehouses is a tedious task and sometimes food present inside the warehouse gets unnoticed and thus get contaminated. Also safety of warehouses from intruders can be done by a robot, in any condition where it is difficult for human beings. This robot would be cheap and efficient and also make sure of safety, keeping the food intact and ensuring its fine quality.
1. Introduction
Since the time living creatures like animals and human beings have come into the picture, food has been one of the core parts of their survival. Having wholesome food is a very basic need for every human being which should include good nutritious food that makes individuals healthy and feel vibrant. There are different types of food that can be consumed by human beings such as it vegetables, fruits, meat, and pulses, and their quality may vary. Quality is the main factor in the food industry because the finest the food quality is, the higher the chances of success of the industry in the competitive market [1] and its demand automatically goes up, along with the increase in consumer awareness nowadays. So, the expectations in terms of food quality had increased the need for monitoring the quality of food [2]. According to a report in the USA (shown in Figure 1), it was found that around $50–80 billion USD cost goes into illness treatment caused by food per year. In India, in 2019, it is around $15 billion annually and in 2020, it is around $28 billion (Rs 1,78,100 crore), as per the statistics by the World Health Organization (WHO). It was found that 1 out of 10 people fall ill due to food they consume which is not of the quality; as a result, we lose around 33 million in the world. This gives the reason why we need to focus on the quality of food, which begins from the very first step, that is, where they are produced and stored (safety of food and food warehouses) [4].
In some food industries, there are trained food inspectors which inspect the food quality manually which makes the inspection monotonous and slow, requires a lot of human effort, and is costly [5]. In numerous cases, it is found that a food inspector during an inspection of food falls ill, because of quality (worsening of food), foul smell, or tasting those food [6]. These industries have different sizes of warehouses and even some small vendors also have huge warehouses to store the foods, so ensuring safety manually is a big challenge to both industry (in monetary terms) and food inspectors (in terms of their health) [7]. So, we have to discover some methods that can decrease the risk of health issues and ensure the safety of food and food warehouses; for example, we can use a robot, which will have some sensors for identifying the quality of food or detecting the food and sending pictures of it; it can be used to protect the warehouse from intruders as well. Robots can be trained by using some artificial intelligence techniques/machine learning algorithms. So, robots can be used to determine the food and give better results in the food industry [8].
Artificial intelligence is basically mimicking functionality like that of a human brain. It includes subsets like deep learning and machine learning, which performs various tasks that need human intelligence like speech recognition, decision-making, and translating languages [9]. Artificial intelligence is used in the food industry (as shown in Figure 2) for various purposes such as food sorting, supply chain management, food preparation, food quality improvement, and food safety [10]. Many companies and government agencies are using these models for making predictions and also they have high efficiency and can operate difficult tasks such as food safety and food quality control [11]. There are a lot of problems that are still not addressed by using artificial intelligence alone [12]. Robots can be trained by using machine learning algorithms to solve the problems mentioned above. These robots can be positioned with the camera to ensure food safety such as monitoring the food condition and preventing intruders. Algorithms such as artificial neural networks (ANN) and convolution neural networks (CNN) are used in the food industry to solve challenges; for example, ANN is one of the best tools for examining food quality as well as food safety. In fruit sorting, wine bending, or other domains we can use ANN to ensure the quality of food. Since the purpose of a food warehouse is to store the food for future use, there are many things that we have to maintain in the warehouse for better food quality. There are a lot of challenges faced in the warehouses such as food storage, food safety, and safety from intruders. Robots can be used to address these problems and have great scope in the field of food industry and they can work in every condition without any interruption and even in the worst conditions [13,14], so the robots can be used to ensure the security of warehouses and food. Robot’s ability to learn quickly and perform repetitive actions in any situation would be of great use [3]. This paper aims to find possible solutions using robots (VIBHISHAN 2.0) to solve the problem of food safety and food warehouses, as it can decrease the efforts and from time to time keep an eye on safety.
2. Literature Review
As shown in Figure 1, about the statistics of food-related diseases which lead to health issues or even death in some cases, we can infer that there is a need to find a solution that can improve the food quality or control the quality. There are many solutions available in the market from 2004 till now. In 2004, a paper mentioned the use of image processing methods in the food industry to give better results. The paper mentioned the applications of image processing techniques for food quality evaluation, in which they suggested methods to evaluate the food quality by capturing the images of food using a camera or MRI. The captured image is then improved in quality to find the distortion or some more features that are used for further processing. This method was cheap and fast to evaluate the food quality [5]. In 2006, “Learning Techniques Used in Computer Vision for Food Quality Evaluation: A Review” was a paper that gave a better estimation method using computer vision. There are many algorithms like ANN, Supervised Learning, Genetic algorithm, decision tree, and fuzzy logic that can be used to evaluate the result in terms of quality, in which genetic algorithm and decision tree algorithm give better results than other algorithms for food quality evaluation [1]. In 2007, a device named “electronic nose” was launched. It was similar to the human nose which was used for detection of food quality. It was based on sensors and was more beneficial than human sensory panels because it evaluated a very large number of samples and gave a more stable and consistent result [7,15,16]. After that, another electronic device was built named “electronic tongue” that gave a better detection and sensing capability than humans as it could also detect the odour of food and all dissolved compounds [17]. In 2008, a system for security was proposed that controlled the monitoring using some sensors and actuators. It had two-level communication systems in which one was used to communicate with modules and the other connected with the device for controlling processes [18]. In 2011, data mining techniques were more used in quality checking and for prediction or optimization processes. So, those techniques were used in food quality checking and making electronic devices for the food industry which made a big contribution in the food industry along with the ANN Algorithm that performed as an optimal process for improvement in food quality output [19].
In 2012, “Achieving Quality Assurance Functionality in the Food Industry Using a Hybrid Case-Based Reasoning and Fuzzy Logic Approach” was a paper that gave a solution for monitoring in a warehouse, in which a system IFQAS was made to aid and evaluate the safety as it used a decision-making process, in which CBR and FBR were the decision support technologies used [20]. In 2013, there was a device for grading the egg based on defect or size, which was done by using the fuzzy logic inferences and computer vision system, used to find the eggs with spots or cracks. It was possible using the designed algorithm in which they grade the eggs as 100% if they were normal, and if they contain blood spots, then the rate was graded as 83% [21]. In 2016, “Prediction of Banana Quality Indices from Color Features Using Support Vector Regression” is a paper which used Support Vector Regression (SVR) implementation and was used for quality prediction of bananas. Also the ANN model was used to give outcomes that were compared with the SVR outcome [22]. In 2017, a smart kitchen was developed named “Foodie” that provided many services for preparing a meal. It gave updates about diet goals or for recipe recommendation and also updates about preferences for better results. It used text or voice recommendation to provide better facility to the user in the field of AI [23]. In 2018, there was a paper for the safety and storage application of warehouses using robotics and Internet of Things (IoT) platforms which may produce an ideal solution for the management of warehouses and also reduce the storage efficiency or time with minimum distance coverage by robot [24]. In 2020, there will be many models developed for the food industry. One was related to bread quality improvement by using wheat germ which decreases the softness of bread throughout the storage process and it gave a specific volume to the bread [25]. Also in 2020, there was another model that was proposed on the basis of the artificial intelligence cloud project, which developed a smart refrigerator and it could send an alert to the user, if the food gets stale or the food is kept in the fridge for many days [26]. In 2020, another system was developed named “An Autonomous Food Wastage Control Warehouse.” It gave four major features using blockchain and various algorithms, in which the warehouse gave priority to the food which was kept first into the warehouse and tried to dispatch it before it turned into stale food [27]. In 2020, “Alternative Data Mining/Machine Learning Methods for the Analytical Evaluation of Food Quality and Authenticity” provided the use of various algorithms to evaluate the food quality, in which SVM, random forest, and regression tree algorithm were used in the area of food quality [28]. In 2021, “Deep Learning Based on Residual Networks for Automatic Sorting of Bananas” was published which gave an automatic mechanism to sort bananas on the basis of images.
3. Problem Statement
Food is the most crucial part of the nourishment of human life. Survival without shelter is still possible but without food is not possible for more than 10 days. So, food plays a very important role in human life as it provides nutrients, fats, minerals, vitamins, and energy to the body and so forth, but it can only be engrossed by the fine quality of food only, not by the food which is of poor quality or is stale/rotten. To maintain the quality of food, it is most important to keep a track of the very first step where the issue occurs, that is, warehouses (the place where food is stored). But food safety is a major concern especially in large warehouses where manual ways of ensuring safety are not feasible. Safety can be from an intruder trying to affect the food stored in a warehouse or food getting stale due to lack of attention on what kind of food is stored or shelf life of food; it causes a lot of food wastage [29]. Also, the government is trying to address this issue of safety for food and warehouses but there is a lot of scope in improving the technology for better transparency and monitoring systems [13]. Our traditional warehouse lacks a system that could provide safety to food and warehouses. So, our main goal is to develop a robot that can give us better results for food safety, which can be integrated with technology like artificial intelligence or machine learning.
4. Hardware Components Used
4.1. Ultrasonic Sensor
It is an electronic instrument that is used to evaluate the distance between the objects by transmitting the ultrasonic wave which gets reflected from the target object and converts the sound waves into electrical signals. For this, it uses the transducer to send the signals and the receiver to receive the signals. It is used as proximity sensors and it has a higher frequency than the human frequency. Sensors are used for object detection or people detection or for navigation. Ultrasonic sensors (as shown in Figure 3) were used to calculate the distance between the target objects by using the distance formula, that is, where D is the distance from the object, T is calculated time by sensors (i.e., signal transmission time and receiving time) and C is the speed of the signal, that is, 343 metres/second.
Here, to ensure the safety of warehouses from any intruder, the robot is positioned with an ultrasonic sensor as it will detect the distance between the intruder and from where the VIBHISHAN (robot) is capturing the image of the intruder. This distance is used by the robot to make its movement according to the distance between intruders.
4.2. Passive Infrared Sensor
PIR is a sensor that is used to detect the motion of an object when it passes into the range of the sensor. It is made up of pyroelectric sensors in which passive means it does not generate any energy in the detection of motion and it detects the amount of infrared radiation, in which if a warm object comes into the range of the sensor, it detects the positive differential range in one slot of the sensor and if it leaves the range of the sensor, then it detects negative differential change into another slot. So, if any human or animal passes through the range of the sensor, then it detects the energy and converts it into electrical signals. Figure 4 demonstrates the image of the PIR sensor, to ensure that the intruder is actually present in the warehouse; that is, some person (unauthorized) present in the warehouse is detected by this sensor.
4.3. Raspberry PI
It is a single computer board connected to devices such as a mouse and keyboard and used as a mini computer. It is used as a web server and to host websites in which we use a small SD card for storing purposes and also used for real-time processing of image and video, or in robotic applications and in IoT-based applications. It is useful for making robots or gaming consoles, tweetbot, or building a server and is also used for automating the home.
It is the core component of a robot as it integrates all the sensors together and collects and processes the data collected from all the sensors, which is shown in Figure 5.
4.4. ESP8266 WIFI
ESP8266 is a very low-cost device used for Internet connectivity. It is used to fetch the data easily and also for uploading it to servers using the Internet. It has a maximum voltage of 3.6 V. It also works as a hotspot or Wi-Fi to provide Internet connectivity to the hardware device in the field of the Internet of things. It is used to send the data in the form of location, image, or video to databases. It can be used for providing the local Internet connectivity to the device where the Internet is necessary to use. Figure 6 demonstrates the pin and labels which is present on ESP8266 WIFI.
Transferring the data collected and processed by the raspberry pi to the owner of the warehouse, so that the owner can take necessary action at the earliest, is done with the help of Internet connectivity provided by WIFI.
4.5. ESP32-CAM
ESP32-CAM is a small camera device for monitoring, recognition, capturing videos and images, and so forth. It can contain a small SD card to store the video or images into it. It can help to detect the intruder using video streaming. It is integrated with the OV2640 camera and several GPIOs to connect peripherals and can store the images clicked by the camera. It is a cost-effective device in hardware for monitoring purposes including both a video camera or SD card for storing the images.
Capturing the images of the intruder, present in the warehouse, and ensuring the safety of food present inside the warehouse are done with the help of a camera installed on the robot. It will help the owner to make decisions faster. Figure 7 shows the sample image of the hardware used in the robot.
5. Working Procedure
A warehouse is required to store the food for the supply chain management. Many basic operations are performed in warehouses like receiving products, ensuring storage allocation, and ensuring the safety of food or transportation [30]. Since it is very difficult to track or monitor huge warehouses by manual efforts only [19,31], there is a need to improve the monitoring techniques for food and warehouse and also ensure the safety of food as well as warehouse. To solve this issue, we designed a robot (VIBHISHAN 2.0) which can detect the intruder and send an alert to the owner/farmer for the safety of the warehouse and it can also check the food and provide images of it (which is at the stage where it can infect other food and quality of food) to the owner/farmer so that they can take necessary action with less human interactions. Three functions are performed by the robot as follows.
5.1. Flow for Intruder Detection
To ensure the safety of the warehouses, a bot/robot named “VIBHISHAN 2.0” is used for intruder detection. Only authorized people should be able to access the warehouse. This bot can be used for this objective, which can be achieved by surveillance. In case the bot is not active, then it is in power saving mode, as shown in Figure 8.
If the robot is active, then, with the camera (ESP32-CAM) located on it, it will record videos and take pictures of the surroundings of the warehouse. During surveillance, it monitors the complete surroundings of the warehouse, as the camera has the ability to rotate 360 degrees. If it does not detect any stranger, it continues surveillance. If anything is detected, then it sends an alert to the owner/farmer that an intruder is detected and sends a picture of it on a mobile application. In case the battery is low or less than 15% or after 15 to 20 minutes of inactivity, it automatically goes into the power saving mode. In this way, the robot can ensure the safety of the warehouse and you will get complete information on mobile applications.
5.2. Flow for Food Safety
Food stored in warehouses should be of fine quality, to make sure this is a major concern. If the food gets stale or rotten, then there is no benefit of putting food into the warehouse as it can also rot other food products. So, there is a need to check the food from time to time and find the status of food to ensure safety.
Our robot also detects the food that is in good condition or not (got stale). As shown in Figure 9, if the bot is not active, then it is in power saving mode; otherwise, it will activate. In the active state, it goes to the warehouse and detects the food with the help of a camera. If food is in good condition, then it continues the food detection and in case the food is rotten or stale, then it sends an alert with the picture of stale food to the owner on the mobile application. If the battery is low or less than 15% or after 15 to 20 minutes of inactivity, it automatically goes into the power saving mode or it is active; then, it waits for further instructions from the user. In this way, we can ensure the safety of food, which is present inside the warehouse.
5.3. Flow for Image Processing
The above two processes demonstrate how we used robots/bot for the safety of food warehouses and food. The primary concept for security/safety is done by using image processing. Raspberry pi is used to load the trained model on a bot, which is shown in Figure 10.
We had trained our machine learning model for detecting the food by using CNN. So, to train our model, we have used both kinds of images for food products, that is, fine quality and stale/rotten. Figure 10 shows the output of the images which are of fine quality. After training the model, it gives maximum accuracy of 96.30%, which is depicted in Figure 11.
Training the model used in the robot VIBHISHAN consists of 4 steps, used to ensure the safety of food by detecting the food (using image processing). The 5 steps are as follows:
Step 1. image acquisition.
Images were taken from a dataset with an image having a 1024 × 1024 resolution, which was then reduced to 512 × 512 resolution. Dataset consists of images analysed: apple, banana, beetroot, paprika, onion, peas, pear, orange, potato chip, mango, hawthorn, tomato, doughnut, peanut, and so forth.
When the image is captured through the camera installed on the robot, then it will help in further steps to identify the food captured and draw useful insights from it.
Step 2. image segmentation.
Once the images are picked up, we explore different regions of images (containing objects and background). RGB images are used in real applications but the color components of the images are converted to black and white for training purposes, that is, monochrome images. Figure 12 demonstrates the conversion of coloured image to monochrome image.
(a)
(b)
(c)
(d)
(e)
(f)
Step 3. shape detection.
Shape detection was the keep step while training our model because describing the features (shape) of an object plays an important role in object classification. The simple shape detection method which detects the complete shape of the food was used. It was based on binary morphological operations. Figure 13 demonstrates how the shape is detected.
Step 4. food recognition.
Once all the above steps are completed, the last step is to recognize the food, which can be tested by the validation dataset. Once it recognized the food correctly, then it was also tested on the robot which captures the images. Figure 14 shows the CNN model.
Figure 15 depicts the whole methodology on which the bot works. The bot is used for both the purpose, that is, intruder detection (safety of warehouse), and food safety.
6. Result
VIBHISHAN is an automated robot that can recognize the intruder and also identify the food condition (of stale or rotten food), which ensures the safety of the warehouse as well as the food stored in it. Figure 16 illustrates the picture of a robot (VIBHISHAN 2.0) which we had made for safety purposes. In addition, it is an efficient and cost-effective robot, which can be used to solve the issue of safety of warehouses and food. It is very small in size yet it performs all the tasks necessary to ensure safety.
As shown in Figure 16, the robot has a camera installed in front of it, which will capture the images of the surrounding of the warehouse; they will be sent to the owners on their mobile apps with the help of Wi-Fi which is present inside the body of the robot. With these wheels attached to the robot, it helps it move and do proper surveillance of the food warehouse and food.
Figure 17 is an output of how this robot identifies the food by using a machine learning algorithm with the help of a camera installed on it. This process will help in reducing the wastage of food or preventing the food from getting stale in large warehouses which get unnoticed by humans.
7. Conclusion
As the food industry heavily depends on warehouses to meet the demand of the increasing population in the world, the safety of food warehouses and food is an essential aspect in industry, to ensure that the quality of the food remains intact, because the slightest breach in safety may lead to contamination of the food present in the warehouses. So, the robot VIBHISHAN 2.0 could be one of the possible solutions to set the seal on the safety of both warehouses and the food present inside it, which could be an alternative to a manual way of ensuring safety or it can also be used with a blend of manual and robot techniques. By this, we can ensure security that would not leave any scope for breach in the safety of food. This robot is cheap and efficient as it can work in all possible scenarios and contribute towards the goal of giving fine quality food to people across the world at minimum cost. It will be the one stop solution to ensure safety of food warehouses and food.
Data Availability
The data used to support the findings of this study have been deposited at https://drive.google.com/drive/folders/1YuKAxOtAggSKn5ap7GI2VQqePgzG07R4?usp=sharing. Coding is available for this article whenever required.
Conflicts of Interest
The authors declare that they have no competing interests.
Authors’ Contributions
All the authors contributed equally and significantly in writing this article and read and approved the final manuscript.