Deep Learning-Based Interpretation of Human Actions and Expressions

Abstract

Human activity recognition is of high significance in many areas such as biomedical engineering and video game developing, sports analytics. As systems today mostly classify physical point-and-click activities based on sensors or video the not yet processed off-line(labeled) streams, there is more to human activity recognition where you can infer deeper into emotional states. In order to do this, the project further augments activity recognition with real-time emotion recognition and behavior summarization into a single vision system. The onboard camera module captures live videos, so it can be detected whether a physical activity and facial expressions are alive. The motion data from video stream are used to detect activities such as standing, sitting, walking, running, squatting down, standing up and going upstairs. Activity classification and transitions are dealt using LSTM networks in a real time manner uniformly. We will revisit the two-fold goal of this system to accurately classify (with CNNs – Convolutional Neural Networks) happiness, sadness, anger and neutrality on facial expression labels. By integrating emotion as well with activity detection, This project achieves real time text summaries for behaviour. Experimental results demonstrate a boost, in the substantially cross-cut communicating with Activity recognition platform-platform-balanced with single, solid photo ingest sensor and is achieved using multi-modal approach⋅ as reported.