Automated Health Monitoring Through Emotion Identification

Abstract

Emotional health refers to the overall psychological well-being of a person. Prolonged disturbances in the emotional state of an individual can affect their health and if left unchecked could lead to serious health disorders. Monitoring the emotional well-being of the individual becomes a vital component in the health administration. Speech and physiological signals like heart rate are affected by emotion and can be used to identify the current emotional state of a person. Combining evidences from these complementary signals would help in better discrimination of emotions. This paper proposes a multimodel approach to identify emotion using a close-talk microphone and a heart rate sensor to record the speech and heart rate parameters, respectively. Feature selection is performed on the feature set comprising features extracted from speech-like pitch, Mel-frequency cepstral coefficients, formants, jitter, shimmer, and heart beat parameters like heart rate, mean, standard deviation, root mean square of interbeat intervals, heart rate variability, etc. Emotion is individually modeled as a weighted combination of speech features and heart rate features. The performance of the models is evaluated. Score-based late fusion is used to combine the two models and to improve recognition accuracy. The combination shows improvement in performance.