Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate

https://doi.org/10.1016/j.bspc.2013.05.010Get rights and content

Highlights

  • An instantaneous heart rate detection method using a low-cost webcam is proposed.

  • A wavelet transform is employed to filter the raw photoplethysmographic signal.

  • The respiration signal is remotely extracted from the heart rate series.

  • We examine performances under motion experiments on 12 healthy subjects.

  • Our technique showed very high agreement (r > 0.85) with reference (BVP) measurements.

Abstract

Photoplethysmographic signals obtained from a webcam are analyzed through a continuous wavelet transform to assess the instantaneous heart rate. The measurements are performed on human faces. Robust image and signal processing are introduced to overcome drawbacks induced by light and motion artifacts. In addition, the respiration signal is recovered using the heart rate series by respiratory sinus arrhythmia, the natural variation in heart rate driven by the respiration. The presented algorithms are implemented on a mid-range computer and the overall method works in real-time. The performance of the proposed heart and breathing rates assessment method was evaluated using approved contact probes on a set of 12 healthy subjects. Results show high degrees of correlation between physiological measurements even in the presence of motion. This paper provides a motion-tolerant method that remotely measures the instantaneous heart and breathing rates. These parameters are particularly used in telemedicine and affective computing, where the heart rate variability analysis can provide an index of the autonomic nervous system.

Introduction

Recognizing an emotion by its physiological signature is a field of research that presents a particular interest on the last 10 years. Understanding emotions can be useful, particularly in virtual therapies, where emotions are feedbacks that regulate the virtual environment level and intensity. Physiological parameters like heart rate (HR) and heart rate variability (HRV) are reliable inputs to emotion recognition [1], [2], [3]. However, contact sensors can be limited in some scopes of application where a specialist must install and monitor them. When dealing with serious games, contact sensors can disturb the interaction and may be intrusive to the privacy.

Non-contact measurements of physiological parameters can be achieved using thermal infrared imaging, a technology employed by Pavlidis et al. to collect physiological data on human faces like heart and respiratory rates, perspiration, supraorbital and periorbital blood flow [4], [5], [6], [7]. Similarly, Doppler radars are non-contact sensors that were used to detect heartbeats [8], [9] and respiration signals [10]. More recently, digital cameras and webcams were employed on the face to detect blood volume pulse (BVP) [11], [12], [13], [14], [15] and compute HR and breathing rate (BR). The principle, based on photoplethysmography (PPG) consists in observing light variations on the skin to recover the cardiovascular pulse wave. This optical technique is mainly implemented in contact pulse oximeters where infrared wavelengths are employed to detect the pulse wave. Considered in this case as noise, ambient light is now an illumination source used for PPG exploitation via high sensitivity cameras and webcams. The main drawback of this technique is that PPG signals are susceptible to motion-induced artifacts, particularly when dealing with webcams and ambient light. Independent component analysis, a blind source separation method, has been proposed by Poh et al. [13] to remove noise artifacts from face imaging PPG signal. Standards of measurement recommend the use of ECG sensors to measure HRV [16]. However, it has been shown that pulse rate variability derived from PPG signals can be a good surrogate of HRV at rest [17]. Sun et al. [14], [15] have compared performances between a low-cost webcam and a high-sensitivity camera to assess HR and pulse rate variability. They conclude that the functional characteristics of a 30 fps webcam are comparable to those of a 200 fps camera when interpolating signals to improve the time domain resolution [15].

The HRV is a parameter used in affective computing and psychophysiology to give an index of the autonomic nervous system (ANS) activity in order to detect workload changes in real time [19]. Its spectral analysis can provide the sympathovagal balance, a ratio that reflects reciprocal changes of sympathetic and vagal outflows [18]. The HRV tends to be rhythmic and ordered in positive emotional states and follows the respiration by a phenomenon called respiratory sinus arrhythmia (RSA). In contrast, the HRV tends to be chaotic and disordered in states of anger, anxiety or sadness. These rhythmic variations provide a state known as cardiac coherence [20], [21]. The use of non-contact means to detect physiological signals is particularly advantageous in affective computing, where the objective is to induce emotions like stress or fear for example. In these psychophysiological experiments, contact sensors may generate a bias by interfering with the user, resulting practically by an erroneous emotion classification [4]. Currently, published methods effectively recover HR, BR and HRV spectral components over a given time period. However, few attempts [15] have been made to measure the instantaneous HR (iHR) with a webcam, especially when considering head motion artifacts.

The immediate objective of this study was to provide a motion-tolerant method that reliably recovers the instantaneous pulse and breathing rates using a low-cost webcam. The presented methodology was developed to overcome signal variations generated by natural head movements. Firstly, we describe the approach, where robust image and signal processing are introduced to gather exclusively skin pixels that contain PPG information. A wavelet filtering algorithm was elaborated to recover both the instantaneous HR and BR. Secondly, we validate the accuracy of the proposed approach using approved contact probes. Webcam PPG signals were remotely recorded from 12 healthy volunteers during a set of two experiments, specifically at rest and during motion. Remote measurements of the instantaneous heart and breathing rates were respectively compared to those acquired from BVP and chest belt sensors.

Section snippets

Experimental procedure

Two experiments were conducted indoors to evaluate the iHR assessment method on 12 healthy volunteers (Table 1) of both gender and various ages. The skin type reported in Table 1 corresponds to a visual estimation of the participant skin color using the Fitzpatrick chromatic scale [22], defined between I for white skins and V for black skins. For practical purposes, categories I and II were regrouped in one set. All participants gave their informed consent before the beginning of a session.

Results

Image and signal processing detailed in the previous section were employed to compute the iHR series and respiration traces of all subjects. These two physiological parameters were simultaneously recorded by contact sensors. A typical example from subject 3 (male, age = 22 years and skin type = I–II) is presented in Fig. 6 where respective signals are directly comparable. Pearson's correlation coefficients along with a statistical analysis and Bland–Altman plots were used to quantify the level of

Discussion

The HRV is a physiological measurement used in several domains, especially in affective computing and personal health care. The present study demonstrates that the instantaneous HR can be assessed robustly using a low-cost HD webcam on human faces, even in presence of motion. Statistical and beat to beat analysis reveals that webcam-derived HR and BR are in close agreement with reference sensors. The respiration signal is recovered using the method presented in Section 2.3.3.2. The breathing

Conclusion

This study presents image and signal processing techniques to remotely assess the instantaneous HR. Using a skin detection filter, the proposed method selects only skin pixels that contain PPG information. The u* component of the CIE L*u*v* color space is used to increase robustness on motion and light variations. A typical example is presented in Fig. 9. A wavelet-based filtering operation is then applied to detrend and denoise raw signals. It has been shown that trends in PPG signals must be

References (33)

  • J. Fei et al.

    Thermistor at a distance: unobtrusive measurement of breathing

    IEEE Transactions on Biomedical Engineering

    (2010)
  • I. Pavlidis et al.

    Thermal image analysis for polygraph testing

    IEEE Engineering in Medicine and Biology Magazine

    (2002)
  • D. Shastri et al.

    Imaging facial signs of neurophysiological responses

    IEEE Transactions on Biomedical Engineering

    (2009)
  • G. Lu et al.

    Contact-free measurement of heartbeat signal via a Doppler radar using adaptive filtering

  • V. Vasu et al.

    Signal processing methods for non-contact cardiac detection using Doppler radar

  • C. Gu et al.

    Doppler radar respiration measurement for gated lung cancer radiotherapy

  • Cited by (0)

    View full text