Development of Smart Emotion Recognition System for Companion Robot

Abstract

The first stage of this study is to establish a convolutional neural network based image recognition model to identify the face image of “Happiness”, “Anger”, and “Sadness”. To further improve the model’s recognition accuracy, the second model is established via adding the collected physiological data such as the heartbeat and body temperature, to form a physiological data-assisted psychological recognition system. The Googlenet is selected as the first model and the long short-term memory (LSTM) and backpropagation neural network (BPNN) are chosen as the second model in this study. By importing the captured image and the detected physiological data, one of following six emotions—Happiness, Anger, Fear, Sadness, Surprise, and Disgust can be assessed. Some simulation results are provided in the first stage. The second stage of this study is to make the system lightweight furtherly, both of the models’ formats are converted and embedded into Raspberry Pi smart control board. Finally, the control board, camera and physiological sensors are equipped into a companion robot to implement the smart object.