Abstract

A face reveals a lot of information about a person's identity, age, sex, race, and emotional as well as psychological state. Facial expressions are often used in the behavioral interpretation of emotions and play a key role in social interactions. Due to its potential applications such as HCI, behavioral science, automatic facial emotion detection is one of the most intriguing and challenging areas in computer vision. Our Facial Emotion Recognition system performs detection and location of faces in a cluttered scene, facial feature extraction, and facial expression classification. Numerous models were experimented such as decision trees and SVM’s before arriving at a Convolutional Neural Networks (CNN) model. Due to large number of filters, CNNs are efficient for image recognition tasks since they can capture spatial features of the inputs. After observing unbelievable performance with deep learning models, we used deep convolution features to better represent the given image instead of using the traditional handcrafted features. We have proposed and developed a Deep Convolutional Neural Network for classifying human emotions from dynamic facial expressions in real-time. Kaggle’s FER-2013 dataset with seven facial emotions labels as happy, sad, surprise, fear, anger, disgust, and neutral is used to train the model. An overall training accuracy of 92.86% and test accuracy of 64.42% are achieved. Finally, a live video stream connected to a face detector feeds images to the neural network. The network subsequently classifies an arbitrary number of faces per image in real-time simutaneously, wherein appropriate emotions are displayed over the subject’s faces and their probabilities are displayed over a real-time bar graph on a separate window.