Towards recognizing facial expressions at deeper level: Discriminating genuine and fake smiles from a sequence of images

Understanding human's emotions is an important task and has application in a variety of fields. Because of that, facial emotion recognition or facial expressions recognition (FER) has gained many attentions of researchers, with different methods proposed, by using multiple sensors, using applying vision approaches from conventional FER to a deep-learning-based system. Although those methods have succeeded in recognizing facial expressions by analyzing the image or combining sequences of frames then concatenate with the audio extracted from a video, however, recognizing real emotion at a deeper level is still a challenge. We can detect a person is smiling, yet to say whether that smile is spontaneous or frustrated is difficult even for us, human. This paper focuses on the study of existing FER methods in discriminating real from fake smiles to get closer to detect deep emotion of a person from a given video. By the end of the paper, we conduct experiments of several models, the best of which uses bidirectional LSTM with attention mechanism on a combination of representations of a face image, gives 98% accuracy on MAHNOB database. The model was tested on SPOS and MMI and gave 87% and 97% accuracy respectively. © 2019 IEEE.