Title: Ecological Factors of Emotion Recognition Using Physiological Signals

Ecological Factors of Emotion Recognition Using Physiological Signals

Delbert Hung*, Dr. Elaine Biddiss*ⱡ, Dr. Tom Chau*ⱡ

* Institute of Biomaterials and Biomedical Engineering, University of Toronto

ⱡ Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital

Abstract:

Research Question: Are physiological responses from emotional experiences modulated by: 1) activity level; 2) stimulus modality and 3) individual differences in reactivity to emotional experiences?

Target Population: The primary focus of development for such technology is those with limited or deficient communication pathways in terms of the expression of emotion (e.g. fear and disgust in Huntington’s or Parkinson’s disease patients).

Introduction/Background: Research activity in the field of emotion recognition has increased in recent years from the healthcare community. Machines that are ‘emotionally aware’ have been sought for the purpose of enhancing user interaction with computers and for telemedicine applications. Literature in emotion recognition using physiological signals has shown feasibility at the conceptual level but is still lacking in studies that address barriers that will arise in practical implementation. A small set of these barriers are reflected in the research questions posed above.

Methods: To investigate these “ecological” factors, we designed a factorial layout experiment with 2 mobility states (sitting/walking) x 3 stimuli modality (visual/auditory/personal imagery) where we will conduct non-invasive surface recording of physiological signals (e.g. electrocardiogram, skin conductance and respiration) from thirty participants. Affective stimuli will be presented from a computer using items from the International Affective Pictures System (IAPS), the International Affective Digitized Sounds (IADS) and personal imagery cues. Emotional responses to the stimuli will be assessed by self-report using a modified Self-Assessment Manikin. Individual differences in emotional experiences will be measured using two psychometric tests: the Affect Intensity Measure and Behaviour Activity/Inhibition Systems Scales.

Features extracted from the physiological signals will be compared in a factorial MANOVA. Emotional experience data from self-reports will be used as labels for supervised machine learning algorithms that will try to classify emotional states based on physiological signal inputs. Multiple machine learning algorithms will be tested and compared to select for one that achieves the highest sensitivity, specificity and overall classification accuracy.

Results and Conclusion: The project is still under the data collection stage and there are no definitive results or conclusions that can be drawn