Human Activity Recognition System

Modern human activity recognition systems are mainly trained and used upon video stream and images data that understand the features and actions variations in the data having similar or related movements. Human Activity Recognition plays a significant role in human-to-human and human-computer interaction. Manually driven system are highly time consuming and costlier. In this project, we aim at designing a cost-effective and faster Human Activity Recognition System which can process both video and image in order to recognize the activity being performed in it, thereby aiding the end user in various applications like surveillance, aiding purpose etc. This system will not only be cost effective but also as a utility-based system that can be incorporated in a large number of applications that will save time and aid in various activities that require recognition process


Introduction
This Project uses video and images dataset for the recognition of the human activity in the assigned dataset. The solution to the problem is availed via neural network architecture. This architecture uses resnet-34 pre-trained on kinetics dataset for processing the videos and it is further refined using the transfer learning on more concentrated activities while using the caption generation technique on the images. It builds a sentence describing the activity in the image dataset named "flickr8k", in which cnnrnn, lstm is used to extract features from image, and text data of caption is used to learn vocabulary to formulate the sentence.
In this project, we intend to design a cost-effective and faster Human Activity Recognition System that can process both video and picture to identify the activity being conducted in it, thus assisting endusers in various applications such as surveillance, helping purpose, etc. This system will not only be cost-effective but also as a utility-based system that can be integrated into a wide variety of applications that can save time and support in different activities that need recognition process and save a lot of time with good accuracy. It will also support blind people to realize what is happening around them by integrating this system into a handheld device.

Objectives
The main objectives of the design of the Human Activity Recognition (HAR) System are: • To understand different models and techniques of Human Activity Recognition Systems based on research papers. • To recognize various Human Activities from video or image data.

•
To provide a cost-effective and faster Human Activity Recognition system. • To provide with a HAR program that can be incorporated into a variety of different realtime applications like surveillance, aiding blind people etc. • To automate the process of Activity recognition from video stream or an image.

Problem Formulation
There has been an ever-rising need for improving the methods involved in Human and Computer Interaction and how computer understands human actions and activities. There is a large population of elderly and blind people out in the world, problem was to help them understand and interact more with their environment and aid them in various activities which led to the problem formulation of human activity recognition system, which can be used in applications beyond aiding elderly and blind, i.e. in video surveillance etc.

Existing System
The existing system was manual where a person had to sit in front of a monitor to monitor and guide human activities, it was hectic, time consuming and costlier system and was prone to human errors and negligence. Further some systems started using sensor data to recognize human activities but they were needed to be worn by the user which limited the scope of activity recognition in open environment in general.

Proposed System
Unlike the existing system, the proposed system takes input in the form of video and image to recognize the activity being performed in it. It is much faster and a cost-effective solution. It uses deep learning to recognize the activities. This system can be used, incorporated or expanded further to cover a wide range of applications. Hence, it acts as a base system for various applications and tasks. Moreover, it can reduce the need of additional staff for entering data. Thereby, reducing the cost of the companies considerably.

Unique Features of the System
The unique features of the Human Activity Recognition system are listed as follows: •

Methodology
A flowchart is a diagram that represents an algorithm, workflow or process. The flowchart shows the steps of various kinds and their order by connecting them with arrows. 8 In the Human Activity Recognition System, data is first passed through the preprocessing function which makes it suitable to fed as input to the neural network, the data then goes through the network which then gives us the output.

User Interface Representation
User interface simulation is a programming methodology utilized by computer device programmers. Today's user interfaces (UIs) are dynamic software elements that play a crucial role in the accessibility of an application. The creation of the UI thus includes not only guidelines and best practice studies, but also a method of growth, including the production of graphic templates and a structured terminology for this visualization. 25 The PyCharm IDE GUI style is used, which is intuitive and user-friendly to suit the habits of everyday usage. The principle is very formal and widely recognized; such that they stress what kind of action the consumer will take.

Results and Discussions
The Human Activity Recognition System gives activity prediction with good accuracy on videos and with decent accuracy on images. Below are some snap shot of training code, done on colab:

Conclusion and Future Scope
Since computer vision is a trending topic in these days, systems like Human Activity Recognition systems is quite useful and effective for solving a variety of application, whether it would be surveillance or monitoring, or aiding the elderly and blind people etc. This not only provide additional comfort to the end-users but also can be deployed into different Organizations in order to reduce the employ workload. The model shows good results on video streams while performing decently on image data. Activity Recognition system are of great importance in modern days due to the convenience and problems which the system offers and solves. Need of Activity recognition for monitoring and surveillance, video segmentation etc is of growing demand in which this system can greatly help. This system can be incorporated in mobiles apps to further aid the elderly and blind people. It is cost-effective and an immense time saving system which is also prone to human errors. This system acts as a base solution for many other applications involving activity recognition. Hence, this system is very beneficial for both individual and organizations for general or specialize purposes.
This project has a tremendous scope in future.
Firstly, video recognition code can be further fined tuned using transfer learning and much bigger datasets can be used to further increase the accuracy of the model. Moreover, web and mobile apps can be built which can call these python scripts via an API call to provide activity recognition on users mobile, and can also aid the elderly and blind people to understand and interact with their surroundings etc and many more real-time applications of activity recognition system.

Acknowledgement
We are very thankful to Dr. Sehijpal Singh, Principal, Guru Nanak Dev Engineering College (GNDEC), Ludhiana, for providing this opportunity to carry out big project research in college labs, which have some of the facilities required for the effective execution of our project. The continuous advice and motivation obtained from Dr. Kiran Jyoti, H.O.D., IT Department, GNDEC Ludhiana, has been of tremendous help in carrying out the project work and is warmly thankful. We would like to convey our sincere gratitude and thank profusely our Project Leader, Dr. Pankaj Bhambri and Project Boss, Prof. Harpreet Kaur, without whose wise advice and encouragement it would have been difficult to complete the project in this way. We would like to express our appreciation to the other faculty members of the Information Technology Department of GNDEC for their academic assistance in this research. Finally, we owe it to all those who have contributed to this report to work both directly and indirectly.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.