Abstract
We propose, in this study, an original method that was developed to remotely measure the instantaneous pulse rate using photoplethysmographic signals that were recorded from a low-cost webcam. The method is based on a prior selection of pixels of interest using a custom segmentation that used the face lightness distribution to define different sub-regions. The most relevant sub-regions are automatically selected and combined by evaluating their respective signal to noise ratio. Performances of the proposed technique were evaluated using an approved contact sensor on a set of seven healthy subjects. Different experiments while reading, with motion or while performing common tasks on a computer were conducted in the laboratory. The proposed segmentation technique was compared with other benchmark methods that were already introduced in the scientific literature. The results exhibit high degrees of correlation and low pulse rate absolute errors, demonstrating that the segmentation we propose in this study outperform available region-of-interest selection methods.
Similar content being viewed by others
References
Humphreys, K., Ward, T., & Markham, C. (2007). Noncontact simultaneous dual wavelength photoplethysmography: A further step toward noncontact pulse oximetry. Review of Scientific Instruments, 78, 044304.
Kranjec, J., Beguš, S., Geršak, G., & Drnovšek, J. (2014). Non-contact heart rate and heart rate variability measurements: A review. Biomedical Signal Processing and Control, 13, 102–112.
Poh, M. Z., McDuff, D., & Picard, R. W. (2011). Advancements in noncontact, multiparameter physiological measurements using a Webcam. IEEE Transactions on Biomedical Engineering, 58, 7–11.
McDuff, D., Gontarek, S., & Picard, R. W. (2014). Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Transactions on Biomedical Engineering, 61, 2948–2954.
Tarassenko, L., Villarroel, M., Guazzi, A., Jorge, J., Clifton, D. A., & Pugh, C. (2014). Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiological Measurement, 35, 807–831.
Monkaresi, H., Calvo, R. A., & Hong, Y. (2014). A machine learning approach to improve contactless heart rate monitoring using a Webcam. IEEE Journal of Biomedical and Health Informatics, 18, 1153–1160.
Kong, L., Zhao, Y., Dong, L., Jian, Y., Jin, X., Li, B., et al. (2013). Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. Optics Express, 21, 17464–17471.
Shao, D., Yang, Y., Liu, C., Tsow, F., Yu, H., & Tao, N. (2014). Noncontact monitoring breathing pattern, exhalation flow rate and pulse transit time. IEEE Transactions on Biomedical Engineering, 61, 2760–2767.
Task Force of the European Society of Cardiology, North American Society of Pacing and Electrophysiology. (1996). Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation, 93, 1043–1065.
Aarts, L. A. M., Jeanne, V., Cleary, J. P., Lieber, C., Nelson, J. S., Oetomo, S. B., et al. (2013). Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit—A pilot study. Early Human Development, 89, 943–948.
Sun, Y., Hu, S., Azorin-Peris, V., Kalawsky, R., & Greenwald, S. (2013). Noncontact imaging photoplethysmography to effectively access pulse rate variability. Journal of Biomedial Optics, 18, 061205.
Healey, J. A., & Picard, R. W. (2005). Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6, 156–166.
Hernandez, J., Morris, R., & Picard, R. (2011). Call Center Stress Recognition with Person-Specific Models. In S. Dmello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective Computing and Intelligent Interaction (Vol. 6974, pp. 538–547). Berlin: Springer.
Hoover, A., Singh, A., Fishel-Brown, S., & Muth, E. (2012). Real-time detection of workload changes using heart rate variability. Biomedical Signal Processing and Control, 7, 333–341.
Melillo, P., Bracale, M., & Pecchia, L. (2011). Nonlinear heart rate variability features for real-life stress detection. Case study: students under stress due to university examination. BioMedical Engineering OnLine, 10, 96.
Yoshino, K., & Matsuoka, K. (2012). Personal adaptive method to assess mental tension during daily life using heart rate variability. Methods of Information in Medicine, 51, 39–44.
Pavlidis, I., Dowdall, J., Sun, N., Puri, C., Fei, J., & Garbey, M. (2007). Interacting with human physiology. Computer Vision and Image Understanding, 108, 150–170.
McDuff, D., Gontarek, S., & Picard, R. (2014). Remote measurement of cognitive stress via heart rate variability. Presented in the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Chicago, USA (pp. 2957–2960).
Bousefsaf, F., Maaoui, C., & Pruski, A. (2014). Remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam. Computers in Biology and Medicine, 53, 154–163.
Stricker, R., Muller, S., & Gross, H. M. (2014). Non-contact video-based pulse rate measurement on a mobile service robot. In IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK (pp. 1056–1062).
Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA (pp. I-511–I-518).
Holton, B. D., Mannapperuma, K., Lesniewski, P. J., & Thomas, J. C. (2013). Signal recovery in imaging photoplethysmography. Physiological Measurement, 34, 1499–1511.
Sun, Y., Hu, S., Azorin-Peris, V., Greenwald, S., Chambers, J., & Zhu, Y. (2011). Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise. Journal of Biomedial Optics, 16, 077010.
Takano, C., & Ohta, Y. (2007). Heart rate measurement based on a time-lapse image. Medical Engineering & Physics, 29, 853–857.
Feng, L., Po, L. M., Xu, X., Li, Y., & Ma, R. (2014). Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Transactions on Circuits and Systems for Video Technology, 25, 879–891.
Lewandowska, M., Rumiński, J., Kocejko, T., & Nowak, J. (2011). Measuring pulse rate with a Webcam—A non-contact method for evaluating cardiac activity. Presented at the Federated Conference on Computer Science and Information Systems, Szczecin, Poland (pp. 405–410).
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008). Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding, 110, 346–359.
Shi, J., & Tomasi, C. (1994). Good features to track. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, USA (pp. 593–600).
Li, X., Chen, J., Zhao, G., & Pietikainen, M. (2014). Remote heart rate measurement from face videos under realistic situations. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Columbus, USA (pp. 4264–4271).
Bousefsaf, F., Maaoui, C., & Pruski, A. (2013). Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate. Biomedical Signal Processing and Control, 8, 568–574.
Mahmoud, T. M. (2008). A new fast skin color detection technique. World Academy of Science, Engineering and Technology, 43, 501–505.
Bal, U. (2014). Non-contact estimation of heart rate and oxygen saturation using ambient light. Biomedical Optics Express, 6, 86–97.
Sahindrakar, P., Haan, G., & Kirenko, I. (2011). Improving motion robustness of contact-less monitoring of heart rate using video analysis. Graduation Project, Eindhoven University of Technology, Eindhoven, Netherland. http://alexandria.tue.nl/extra1/afstversl/wsk-i/sahindrakar2011.pdf.
Verkruysse, W., Svaasand, L. O., & Nelson, J. S. (2008). Remote plethysmographic imaging using ambient light. Optics Express, 16, 21434–21445.
Asthana, A., Zafeiriou, S., Cheng, S., & Pantic, M., Incremental face alignment in the wild. Presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Columbus, USA (pp. 1859–1866).
Poynton, C. (2003). The CIE system of colorimetry. In J. Schanda (Ed.), Digital Video and HDTV: Algorithms and interfaces (pp. 211–231). San Francisco: Morgan Kaufmann Publishers Inc.
Tarvainen, M. P., Ranta-Aho, P. O., & Karjalainen, P. A. (2002). An advanced detrending method with application to HRV analysis. IEEE Transactions on Biomedical Engineering, 49, 172–175.
Fitzpatrick, T. B. (1975). Sun and skin (Soleil et peau). J. Med. Esthétique, 2, 33–34.
Smith, A. R. (1978). Color gamut transform pairs. Presented at the 5th annual conference on Computer graphics and interactive techniques (pp. 12–19). New York: ACM Press.
Hajiarbabi, M., & Agah, A. (2014). Face detection in color images using skin segmentation. Journal of Intelligent and Robotic Systems, 8, 41–51.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Frederic Bousefsaf, Choubeila Maaoui and Alain Pruski declare that they have no conflict of interest.
Ethical Approval
All procedures performed in this study were in accordance with the 1964 Helsinki declaration and its later amendments. Informed consent was obtained from all individual participants included in the study.
Rights and permissions
About this article
Cite this article
Bousefsaf, F., Maaoui, C. & Pruski, A. Automatic Selection of Webcam Photoplethysmographic Pixels Based on Lightness Criteria. J. Med. Biol. Eng. 37, 374–385 (2017). https://doi.org/10.1007/s40846-017-0229-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40846-017-0229-1