Next Article in Journal
Sharing Soil and Building Geophysical Data for Seismic Characterization of Cities Using CLARA WebGIS: A Case Study of Matera (Southern Italy)
Next Article in Special Issue
Processing of EMG Signals with High Impact of Power Line and Cardiac Interferences
Previous Article in Journal
Underwater Excavation Records Using Underwater Acoustic Survey: A Case Study in South Korea
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Non-Contact SpO2 Prediction System Based on a Digital Camera

1
Electrical Engineering Technical College, Middle Technical University, Baghdad 10022, Iraq
2
School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2021, 11(9), 4255; https://doi.org/10.3390/app11094255
Submission received: 30 March 2021 / Revised: 2 May 2021 / Accepted: 4 May 2021 / Published: 7 May 2021
(This article belongs to the Special Issue Novel Advances of Image and Signal Processing)

Abstract

:
Patients with the COVID-19 condition require frequent and accurate blood oxygen saturation (SpO2) monitoring. The existing pulse oximeters, however, require contact-based measurement using clips or otherwise fixed sensor units or need dedicated hardware which may cause inconvenience and involve additional appointments with the patient. This study proposes a computer vision-based system using a digital camera to measure SpO2 on the basis of the imaging photoplethysmography (iPPG) signal extracted from the human’s forehead without the need for restricting the subject or physical contact. The proposed camera-based system decomposes the iPPG obtained from the red and green channels into different signals with different frequencies using a signal decomposition technique based on a complete Ensemble Empirical Mode Decomposition (EEMD) technique and Independent Component Analysis (ICA) technique to obtain the optical properties from these wavelengths and frequency channels. The proposed system is convenient, contactless, safe and cost-effective. The preliminary results for 70 videos obtained from 14 subjects of different ages and with different skin tones showed that the red and green wavelengths could be used to estimate SpO2 with good agreement and low error ratio compared to the gold standard of pulse oximetry (SA210) with a fixed measurement position.

1. Introduction

Eighteen months after the emergence of COVID-19, the number of cases worldwide amounts to more than 152 million people, with more than 3.2 million deaths. In Iraq, the COVID-19 pandemic has infected more than 1 million people and caused more than 15,000 total deaths [1]. COVID-19 causes several symptoms, including fever, cough, fatigue and a respiratory illness whereby the patient often suffers from shortness of breath which causes a decrease in blood oxygen saturation level [2,3]. As cases rise, the terms ‘oxygen saturation’ and ‘pulse oximetry’ regarding COVID-19 infection have been extensively utilised. Pulse oximetry is a non-invasive optical technique based on a photoplethysmography (PPG) signal obtained from the wavelength illumination transmitted or reflected by a tissue that is used to measure blood oxygen saturation (SpO2) level in both hospital and home environments [4]. With the urgent need for this device in hospitals or during home quarantine, it has become a significant concern for researchers to develop it and find suitable alternatives at the lowest possible cost.
Extracting SpO2 is often essential to maintain patients’ health because of its primary role in diagnosing respiratory diseases and in patient monitoring, especially during this global pandemic. Pulse oximetry relies on direct contact, most commonly with a fingertip, which restricts the patient and may become uncomfortable [5]. In addition, it may cause skin damage in subjects with a very fragile skin, especially infants in neonatal care, if it is left attached to the skin for long periods [6]. Moreover, pulse oximetry devices do not always fit well with people’s fingertips, depending on patient’s age and fingers’ size, which may lead to distorted readings [7]. In addition, direct contact with the device exposes people to the risk of infection and skin irritation [8,9]. Other reported limitations that may introduce errors include motion artefacts, and are linked to anaemia, venous pulsations, skin pigmentation, low perfusion, light interference and presence of nail polish [10,11,12,13]. Therefore, there is a need to monitor SpO2 when contact with the skin is either undesirable or unsafe. With the rapid growth of remote-imaging technologies, non-contact and unobtrusive SpO2 monitoring has become feasible.
Colour-based techniques using cameras in a variety of forms (such as digital cameras, thermal cameras, IR cameras, Webcams, drone and Kinect), sometimes called imaging photoplethysmography (iPPG), have been widely used in healthcare for monitoring applications, especially to evaluate vital signs, including heart rate (HR) [14,15,16,17], heart rate variability (HRV) [18,19,20], breathing rate (BR) [21,22,23] and blood pressure (BP) [24,25,26]. The principle of iPPG is the non-contact monitoring of human cardiac synchronous variations by detecting optical features passing through or reflecting from the epidermis captured by a camera sensor. Experiments regarding such techniques, however, were performed under extremely controlled conditions in terms of subject movement, region of interest (ROI), distance and illumination conditions. The advantages and disadvantage of iPGG have been intensively discussed [9,27,28].
Conventional contact pulse oximetry mainly utilises two separate wavelength illuminations, the red light wavelength (660–700 nm) and the infrared (IR) light wavelength (800–950 nm) [29]. SpO2 can be determined as a linear function by considering the ‘ratio of ratios’ (RRs) relationship between the PPG signal in the red over IR wavelengths [29,30].
The disadvantages of contact pulse oximetry have motivated research to use iPPG based on camera sensors (visible light and/or IR cameras) as an alternative. For example, the red–IR wavelengths were used in several studies [31,32,33] to predict SpO2. The possibility of estimating SpO2 using only visible-light camera channels as reference wavelengths instead of IR have been investigated in different studies. For example, the blue (B) and/or red (R) wavelengths have been used instead of IR [8,34,35,36,37]. Another study by Alharbi et al. [5] used the green (G) and orange wavelength illuminations as an alternative to IR to extract SpO2. However, extracting SpO2 using visible light cameras presents additional issues compared to standard pulse oximetry. First, an unclear ROI resulting from movement may distort the reading. Second, using visible wavelengths means more interference with the outer tissues and superficial vasculature than using the longer wavelengths of red and IR. In addition, iPPG is prone to other artefacts, involving surface reflections, shadowing and skin tone [8].
The principal contributions of the proposed system are: (1) the possibility of predicting SpO2 based on RRs obtained from a visible-light camera using green wavelength instead of IR while keeping red wavelength as the saturation-sensitive wavelength, (2) the extraction of SpO2 directly from one camera using two channel components (R and G) and reducing skin tone limitations resulting from the B channel, thus improving the accuracy of SpO2 measurement, (3) the automatic selection of the forehead region based on a skin detection algorithm and the use of a signal decomposition technique based on complete EEMD [38] and ICA [39] techniques to decompose iPPG signals into different frequency signals and to remove noise artefacts embedded in the iPPG signal.
This paper is structured as follows: Section 2 introduces the principle of SpO2 measurement; Section 3 presents the materials and methods, including research ethics considerations, participants and experimental setup and an overview of the proposed system; Section 4 reports and discusses the experimental results of SpO2 predictions with comparison to standard pulse oximetry readings, with statistical results; finally, Section 5 concludes the work presented in this paper.

2. Principle of SpO2 Measurement

The variation in blood concentration due to the amount of oxygen carried by haemoglobin leads to a variation in reflected wavelength illumination captured by a photodiode or camera. This is because of the different light absorption properties of oxygenated haemoglobin ( HbO 2 ) with respect to deoxygenated haemoglobin ( Hb ) that absorbs light at different wavelengths. Under each illuminating wavelength, the absorption of Hb differs from that of HbO2, and by measuring the absorption of the blood at both wavelengths, it is possible to estimate the SpO2 level as follows [40]:
SpO 2   ( % ) = HbO 2 Hb + HbO 2 %
According to the theory of light reflection and the Beer–Lambert law, measuring SpO2 requires two different wavelengths ( λ 1 and λ 2 ) to determine the maximum transmissive/reflective intensity of two-wavelength illuminations, which decays exponentially with the absorption coefficient and the optical path length. The absorption ratio ( K a ) can be estimated from an RRs measurement as follows [5]:
K a = μ ( λ 1 ) μ ( λ 2 ) = A C λ 1 / D C λ 1 A C λ 2 / D C λ 2
where µ is the absorption coefficient, AC is pulsatile absorption, and DC is non-pulsatile absorption. To estimate the SpO2, the absorption coefficients of HbO2 and Hb should differ significantly at one wavelength and be approximately equal at the other wavelength. The absorption spectrum of HbO2 and Hb at different wavelength illuminations is shown in Figure 1.

3. Materials and Methods

3.1. Research Ethics

The study adhered to the ethical standards of the Declaration of Helsinki (Ethical principles for medical research involving human subjects) throughout the research procedures. Ethical clearance was granted by the Human Research Ethics Committee at the University of South Australia (Protocol number: 0000034901). A written informed consent was obtained from all participants and their guardians to capture video recordings at different times, and all participants were informed about the possibility of withdrawing from the study at any time until data analysis. The confidentiality of the participants was maintained by hiding their faces and preserving their privacy.

3.2. Participants and Experimental Setup

The participants in this study were 14, including 11 subjects (6 males and 5 females) aged from 6 to 50 years and 3 male babies (<6 months old) with different skin tones. The video data were recorded using a digital camera (Nikon D5300, Tokyo, Japan, 10 MP, 18–55 mm Lens) with a resolution of 1920 × 1080 and a frame rate of 60 fps. The distance between the camera and the subject was approximately 1.5–2 m. All adults’ participants were asked to be stationary while a video was captured. A pulse oximeter (Rossmax, Model SA210, Taipei, Taiwan) was used as a reference for validation purposes. Figure 2 shows video recordings from some participants using a Nikon D5300 digital camera with the pulse oximeter attached.

3.3. System Overview

In this section, we discuss the image and signal processing techniques that were designed to extract the relevant wavelengths from the selected forehead signal and eliminate noise artefacts embedded in the image spatial domain. These techniques include a face detection method, a signal decomposition technique, the ICA technique, a spectral analysis method and finding local maxima of the extracted signals. The overall system design of the non-contact SpO2 prediction system is illustrated in Figure 3.
After converting the videos into frames, the skin detection algorithm based on skin tone was applied on the first frame to select the ROI automatically. The algorithm returned the bounding box of the skin portion, especially the face, and the ROI was detected automatically based on skin tone. This process was repeated for all frames. In the skin detection algorithm, the RGB image was converted to the LAB device-independent colour space transformation, as this colour space is useful for image segmentation, as shown in Figure 4a. The A (red-green) and B (yellow-blue) components of this colour space (see Figure 4b,c) were chosen and then converted into binary images, as shown in Figure 4d,e. Image binarisation was done based on the gray image threshold histogram method. After that, binary images were multiplied and AND logic was performed on the image, as shown in Figure 4f.
As a result, a matrix was obtained having 1 and 0. One (1) indicated the presence of skin. Thus, facial ROI could be detected automatically, followed by determining face border points to select the forehead, as shown in Figure 3.
After that, the red and green channels could be represented by the brightness of the colour images as follows: [41,42]:
R = E λ S R   d λ ,
G = E λ S G   d λ ,  
where E λ is the light spectrum, and S R and S G are the sensitivity functions of the red and green channels at a wavelength of λ . The average time-series signal of the brightness pixel values for the red and green channels can be expressed as [28,43]
i R   ( t ) = x , y R O I F   ( x , y ) | R O I |
i G   ( t ) = x , y R O I F   ( x , y ) | R O I |
where F ( x , y ) is the brightness pixel value at a location ( x , y ) from the red and green channels, and | R O I | is the pixel area of the selected forehead ROI, where the average value for each channel is in a range from 0 to 255. The average pixel signals from both channels are illustrated in Figure 5.
The next processing stage consisted in using a complete EEMD method [38]. This decomposition technique is commonly used to remove noise artefacts from biomedical signals [44,45,46]. Using this technique, i R   ( t ) and i G   ( t ) were composed into a set of signals with different frequency scales, called intrinsic mode functions (IMFs), as shown in Figure 6 and Figure 7.
In order to estimate oxygen saturation, we used the low-frequency IMF5 from the red channel and the high-frequency IMF2 from the green channel that is representative of the infrared wavelength used in traditional pulse oximetry. Next, IMF5 (Red) and IMF2 (Green) were chosen as inputs to the ICA technique because they provide a low- and high-frequency range with a clear peak as well as the best SpO2 estimation among other IMFs. The schematic diagram of ICA is presented in Figure 8.
ICA is a statistical signal processing technique for solving blind source separation problems using high-order statistics [39] and is widely used to remove noise artefacts from biomedical signals [47,48,49]. The input and output signals of ICA are shown in Figure 9.
Figure 9 (first row) shows 10 s of the raw IMFs signals obtained from EEMD that contain the plethysmographic information. Figure 9 (second row) represents the observed signals after the mixing process. Figure 9 (third row) shows the estimated signals after applying a blind source separation. All power spectra of the raw IMFs signal and ICA components contained visible peaks, but the power spectrum of the outputs of ICA (third row) yielded the highest SNR and the closest estimate of SpO2. Since the output signals of ICA ( u 1 and u 2 ) have different wavelengths, Equation (2) based on RRs measurement was used to obtain the absorption ratio ( K a ). Now, the SpO2 level was calculated using the following equation:
SpO 2 = A B K a
where A and B are empirical coefficients determined by a calibration, whose values were selected as 100 and 20, respectively.

4. Experimental Results and Discussion

In this section, the performance of SpO2 was evaluated for 14 subjects, and the obtained results are reported and compared with the results measured by the Rossmax pulse oximeter. All subjects were healthy due to the difficulty of collecting data from COVID-19 patients. For each participant, we recorded five videos at different times in the stationary scenario. As a result, 14 subjects accounted for 70 video data samples in total. Video processing was performed in a MATLAB environment due to its simplicity that allowed easy application development.
First, to determine why the selected wavelengths from the output signals of ICA were the best to estimate SpO2, a fast Fourier transform (FFT) was performed for spectrum analysis of these signals and to obtain their spectrogram, as shown in Figure 10.
The resulting images from Figure 10 clearly revealed distinct optical properties and the wavelengths of the red and green signals resulting from applying ICA on the selected IMFs.
Second, to determine the correlation rate between the obtained SpO2 data using the proposed method and the measured SpO2 data from the Rossmax pulse oximeter, a correlation plots with some statistics, including Pearson’s correlation, Spearman’s Rho correlation and Kendall tau correlation coefficient, was used, as shown in Figure 11.
As shown in Figure 11, the slope of the correlation plot was −0.0914 with Pearson’s correlation coefficient of −0.074, Spearman’s Rho correlation coefficient of −0.086 and Kendall tau correlation coefficient of −0.058.
To determine the limits of agreement and error ratios between the measured and the obtained SpO2 data, the Bland–Altman method [50], sum of square error (SSE), mean absolute error (MAE) and root-mean-square error (RMSE) were used, as shown in Figure 12.
It is clear that the mean bias and the upper and lower limits of agreement between the percentage of SpO2 readings were −0.57, +6.5, −7.6, respectively, with error ratios of 2.72, 3.02 and 3.62 for the SSE, MAE and RMSE, respectively. It can be observed that the results from the proposed system agreed closely with those from a standard pulse oximetry device and fell within the normal range of SpO2 values.
To evaluate the execution time of the proposed system, the MATLAB code was executed on a laptop with Intel(R) Core ™ i7-6700HQ, CPU of 2.6 GHz with 16 GBDDR4 (2400 MHz) under the Windows 10 operating system. The execution time to process a video (10 s) was 105 s, including 45 s for converting video to frames (600 frames), 55 s for ROI selection and 5 s for SpO2 estimation.
This study has a number of limitations that might need to be overcome to achieve its practical implementation. Video in normal environments is subject to movement and unknown lighting conditions. The amount of direct and ambient light available outdoors or in domestic environments cannot be known when the device design is done, possibly leading to dim and noisy images. The background behind subjects is also uncontrolled, and often affects the video quality of the subject’s face due to automatic gain controls and automatic white balance. These difficulties can normally be managed by the digital interface for camera controls in dedicated computer vision cameras, but for general-purpose video cameras, many of these parameters cannot be disabled.
This study limited itself to the consideration of stationary subjects, yet in many instances, children and some adults simply cannot be forced to remain still. For some types of movement, it may be possible to stabilise the images using software or gimbal-mounted cameras. For other motions, particularly when the subject changes the angle of their heads, stabilisation might not be effective.
Although the image sequences and processing appeared to take a long time, the Matlab environment is not representative of the speed of a dedicated application. A fielded application of the algorithm would process each frame as it arrives, segmenting and processing as quickly as possible. It is likely that an implementation in the C++ programming language would run in real time. Further validation of the results is required. The sample size was very small, even though the theory supports the operation of the system and validated the results that were obtained. A diversity of individuals and skin tones would be desirable, as would a diversity of SpO2 levels amongst the subjects.

5. Conclusions

Currently, pulse oximetry relies on contact-based clips or otherwise fixed sensor units, which may cause discomfort, infection or adverse reactions in patients with sensitive skin. There is a need for a tool that will help people to remotely measure their SpO2, thus decreasing the risk of infection during the pandemic and afterwards. This study presents a new camera-based system to estimate SpO2 using visible-light wavelengths based on RRs measurement. Based on the preliminary results, the proposed system shows similar readings as those obtained by the standard pulse oximeter commonly used; thus, it holds promise as a tool for predicting SpO2 with remote-imaging technologies in a way that is cost-effective and highly available.

Author Contributions

Conceptualization, A.A.-N.; methodology, A.A.-N., G.A.K. and J.F.M.; software, A.A.-N.; validation, A.A.-N.; formal analysis, A.A.-N., G.A.K. and J.F.M.; investigation, A.A.-N., G.A.K., J.F.M. and J.C.; resources, A.A.-N., G.A.K. and J.F.M.; data curation, A.A.-N., G.A.K. and J.F.M.; writing—original draft preparation, A.A.-N.; writing—review and editing, A.A.-N., G.A.K., J.F.M. and J.C.; visualization, A.A.-N., G.A.K. and J.F.M.; project administration, A.A.-N. and J.C.; funding acquisition, J.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Human Research Ethics Committee at the University of South Australia (Protocol number: 0000034901).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study and written informed consent has been obtained from the patient(s) to publish this paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. COVID-19 Coronavirus Pandemic. Available online: https://www.worldometers.info/coronavirus/ (accessed on 1 May 2021).
  2. Struyf, T.; Deeks, J.J.; Dinnes, J.; Takwoingi, Y.; Davenport, C.; Leeflang, M.M.; Spijker, R.; Hooft, L.; Emperador, D.; Dittrich, S. Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19 disease. Cochrane Database Syst. Rev. 2020, 7, CD013665. [Google Scholar] [CrossRef]
  3. Moro, E.; Priori, A.; Beghi, E.; Helbok, R.; Campiglio, L.; Bassetti, C.; Bianchi, E.; Maia, L.; Ozturk, S.; Cavallieri, F. The international European Academy of Neurology survey on neurological symptoms in patients with COVID-19 infection. Eur. J. Neurol. 2020, 27, 1727–1737. [Google Scholar] [CrossRef] [PubMed]
  4. Tamura, T. Current progress of photoplethysmography and SPO2 for health monitoring. Biomed. Eng. Lett. 2019, 9, 21–36. [Google Scholar] [CrossRef]
  5. Alharbi, S.; Hu, S.; Mulvaney, D.; Barrett, L.; Yan, L.; Blanos, P.; Elsahar, Y.; Adema, S. Oxygen saturation measurements from green and orange illuminations of multi-wavelength optoelectronic patch sensors. Sensors 2019, 19, 118. [Google Scholar] [CrossRef] [Green Version]
  6. Herrmann, C.; Metzler, J. Distant Pulse Oximetry. In Laser Doppler Vibrometry for Non-Contact Diagnostics; Springer: Berlin/Heidelberg, Germany, 2020; pp. 167–178. [Google Scholar]
  7. Bui, N.; Nguyen, A.; Nguyen, P.; Truong, H.; Ashok, A.; Dinh, T.; Deterding, R.; Vu, T. Smartphone-Based SpO2 Measurement by Exploiting Wavelengths Separation and Chromophore Compensation. ACM Trans. Sens. Netw. 2020, 16, 1–30. [Google Scholar] [CrossRef]
  8. Guazzi, A.R.; Villarroel, M.; Jorge, J.; Daly, J.; Frise, M.C.; Robbins, P.A.; Tarassenko, L. Non-contact measurement of oxygen saturation with an RGB camera. Biomed. Opt. Express 2015, 6, 3320–3338. [Google Scholar] [CrossRef] [Green Version]
  9. Al-Naji, A.; Gibson, K.; Lee, S.-H.; Chahl, J. Monitoring of cardiorespiratory signal: Principles of remote measurements and review of methods. IEEE Access 2017, 5, 15776–15790. [Google Scholar] [CrossRef]
  10. Abay, T.Y.; Kyriacou, P.A. Reflectance photoplethysmography as noninvasive monitoring of tissue blood perfusion. IEEE Trans. Biomed. Eng. 2015, 62, 2187–2195. [Google Scholar] [CrossRef] [PubMed]
  11. Chan, E.D.; Chan, M.M.; Chan, M.M. Pulse oximetry: Understanding its basic principles facilitates appreciation of its limitations. Respir. Med. 2013, 107, 789–799. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Kyriacou, P.A. Pulse oximetry in the oesophagus. Physiol. Meas. 2005, 27, R1. [Google Scholar] [CrossRef] [PubMed]
  13. Jubran, A. Pulse oximetry. In Applied Physiology in Intensive Care Medicine 1; Springer: Berlin/Heidelberg, Germany, 2012; pp. 51–54. [Google Scholar]
  14. Laurie, J.; Higgins, N.; Peynot, T.; Roberts, J. Dedicated exposure control for remote photoplethysmography. IEEE Access 2020, 8, 116642–116652. [Google Scholar] [CrossRef]
  15. Patil, O.R.; Wang, W.; Gao, Y.; Xu, W.; Jin, Z. A low-cost, camera-based continuous ppg monitoring system using laplacian pyramid. Smart Health 2018, 9, 2–11. [Google Scholar] [CrossRef]
  16. Lee, H.; Ko, H.; Chung, H.; Lee, J. Robot Assisted Instantaneous Heart Rate Estimator using Camera based Remote Photoplethysmograpy via Plane-Orthogonal-to-Skin and Finite State Machine. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2020, 2020, 4425–4428. [Google Scholar] [CrossRef]
  17. Al-Naji, A.; Perera, A.G.; Chahl, J. Remote measurement of cardiopulmonary signal using an unmanned aerial vehicle. IOP Conf. Ser. Mater. Sci. Eng. 2018, 405, 012001. [Google Scholar] [CrossRef] [Green Version]
  18. Benezeth, Y.; Li, P.; Macwan, R.; Nakamura, K.; Gomez, R.; Yang, F. Remote Heart Rate Variability for Emotional State Monitoring. In Proceedings of the 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), Las Vegas, NV, USA, 4–7 March 2018; pp. 153–156. [Google Scholar]
  19. Béres, S.; Holczer, L.; Hejjel, L. On the minimal adequate sampling frequency of the photoplethysmogram for pulse rate monitoring and heart rate variability analysis in mobile and wearable technology. Meas. Sci. Rev. 2019, 19, 232–240. [Google Scholar] [CrossRef] [Green Version]
  20. Nagpal, S. Contactless measurement of Heart Rate Variability by Extracting PPG Signals from Video. Master’s Thesis, Eindhoven University of Technology, Eindhoven, The Netherlands, 30 September 2019. [Google Scholar]
  21. Chen, M.; Zhu, Q.; Zhang, H.; Wu, M.; Wang, Q. Respiratory Rate Estimation from Face Videos. In Proceedings of the 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), Chicago, IL, USA, 19–22 May 2019; pp. 1–4. [Google Scholar]
  22. Luguern, D.; Benezeth, Y.; Moser, V.; Dunbar, L.A.; Braun, F.; Lemkaddem, A.; Nakamura, K.; Gomez, R.; Dubois, J. Remote Photoplethysmography Combining Color Channels with SNR Maximization for Respiratory Rate Assessment. In Proceedings of the 2020 14th International Symposium on Medical Information Communication Technology (ISMICT), Nara, Japan, 20–22 May 2020; pp. 1–6. [Google Scholar]
  23. Luguern, D.; Perche, S.; Benezeth, Y.; Moser, V.; Dunbar, L.A.; Braun, F.; Lemkaddem, A.; Nakamura, K.; Gomez, R.; Dubois, J. An Assessment of Algorithms to Estimate Respiratory Rate from the Remote Photoplethysmogram. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020; pp. 304–305. [Google Scholar]
  24. Holyoke, P.; Yogaratnam, K.; Kalles, E. Web-Based Smartphone Algorithm for Calculating Blood Pressure From Photoplethysmography Remotely in a General Adult Population: Validation Study. J. Med. Internet Res. 2021, 23, e19187. [Google Scholar] [CrossRef] [PubMed]
  25. Chowdhury, M.H.; Shuzan, M.N.I.; Chowdhury, M.E.; Mahbub, Z.B.; Uddin, M.M.; Khandakar, A.; Reaz, M.B.I. Estimating blood pressure from the photoplethysmogram signal and demographic features using machine learning techniques. Sensors 2020, 20, 3127. [Google Scholar] [CrossRef]
  26. Huang, P.-W.; Lin, C.-H.; Chung, M.-L.; Lin, T.-M.; Wu, B.-F. Image Based Contactless Blood Pressure Assessment Using Pulse Transit Time. In Proceedings of the 2017 International Automatic Control Conference (CACS), Pingtung, Taiwan, 12–15 November 2017; pp. 1–6. [Google Scholar]
  27. Sinhal, R.; Singh, K.; Raghuwanshi, M. An overview of remote photoplethysmography methods for vital sign monitoring. Comput. Vis. Mach. Intell. Med. Image Anal. 2020, 992, 21–31. [Google Scholar]
  28. Khanam, F.-T.-Z.; Al-Naji, A.; Chahl, J. Remote monitoring of vital signs in diverse non-clinical and clinical scenarios using computer vision systems: A review. Appl. Sci. 2019, 9, 4474. [Google Scholar] [CrossRef] [Green Version]
  29. Moço, A.; Verkruysse, W. Pulse oximetry based on photoplethysmography imaging with red and green light. J. Clin. Monit. Comput. 2021, 35, 123–133. [Google Scholar] [CrossRef] [PubMed]
  30. Moço, A.; Stuijk, S.; de Haan, G. Posture effects on the calibratability of remote pulse oximetry in visible light. Physiol. Meas. 2019, 40, 035005. [Google Scholar] [CrossRef]
  31. Wieringa, F.P.; Mastik, F.; van der Steen, A.F. Contactless multiple wavelength photoplethysmographic imaging: A first step toward “SpO 2 camera” technology. Ann. Biomed. Eng. 2005, 33, 1034–1041. [Google Scholar] [CrossRef]
  32. Van Gastel, M.; Stuijk, S.; De Haan, G. New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring. Sci. Rep. 2016, 6, 1–16. [Google Scholar]
  33. Verkruysse, W.; Bartula, M.; Bresch, E.; Rocque, M.; Meftah, M.; Kirenko, I. Calibration of contactless pulse oximetry. Anesth. Analg. 2017, 124, 136. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Tarassenko, L.; Villarroel, M.; Guazzi, A.; Jorge, J.; Clifton, D.; Pugh, C. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol. Meas. 2014, 35, 807. [Google Scholar] [CrossRef]
  35. Villarroel, M.; Guazzi, A.; Jorge, J.; Davis, S.; Watkinson, P.; Green, G.; Shenvi, A.; McCormick, K.; Tarassenko, L. Continuous non-contact vital sign monitoring in neonatal intensive care unit. Healthc. Technol. Lett. 2014, 1, 87–91. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Bal, U. Non-contact estimation of heart rate and oxygen saturation using ambient light. Biomed. Opt. Express 2015, 6, 86–97. [Google Scholar] [CrossRef] [Green Version]
  37. Casalino, G.; Castellano, G.; Zaza, G. A mHealth Solution for Contact-Less Self-Monitoring of Blood Oxygen Saturation. In Proceedings of the 2020 IEEE Symposium on Computers and Communications (ISCC), Rennes, France, 7–10 July 2020; pp. 1–7. [Google Scholar]
  38. Torres, M.E.; Colominas, M.A.; Schlotthauer, G.; Flandrin, P. A Complete Ensemble Empirical Mode Decomposition with Adaptive Noise. In Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, 22–27 May 2011; pp. 4144–4147. [Google Scholar]
  39. Hyvärinen, A.; Hurri, J.; Hoyer, P.O. Independent component analysis. In Natural Image Statistics; Springer: London, UK, 2009; pp. 151–175. [Google Scholar]
  40. Kong, L.; Zhao, Y.; Dong, L.; Jian, Y.; Jin, X.; Li, B.; Feng, Y.; Liu, M.; Liu, X.; Wu, H. Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. Opt. Express 2013, 21, 17464–17471. [Google Scholar] [CrossRef]
  41. Tkalcic, M.; Tasic, J.F. Colour Spaces: Perceptual, Historical and Applicational Background; In Proceedings of the IEEE Region 8 EUROCON 2003. Computer as a Tool, Ljubljana, Slovenia, 22–24 September 2003; Volume 1. [Google Scholar]
  42. Al-Naji, A.; Fakhri, A.B.; Gharghan, S.K.; Chahl, J. Soil color analysis based on a RGB camera and an artificial neural network towards smart irrigation: A pilot study. Heliyon 2021, 7, e06078. [Google Scholar] [CrossRef]
  43. Al-Naji, A.; Chahl, J.; Lee, S.-H. Cardiopulmonary signal acquisition from different regions using video imaging analysis. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2019, 7, 117–131. [Google Scholar] [CrossRef]
  44. Xu, Y.; Luo, M.; Li, T.; Song, G. ECG signal de-noising and baseline wander correction based on CEEMDAN and wavelet threshold. Sensors 2017, 17, 2754. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Zhang, X.; Zhou, P. Filtering of surface EMG using ensemble empirical mode decomposition. Med. Eng. Phys. 2013, 35, 537–542. [Google Scholar] [CrossRef] [Green Version]
  46. Al-Naji, A.; Chahl, J. Remote optical cardiopulmonary signal extraction with noise artifact removal, multiple subject detection & long-distance. IEEE Access 2018, 6, 11573–11595. [Google Scholar]
  47. Safieddine, D.; Kachenoura, A.; Albera, L.; Birot, G.; Karfoul, A.; Pasnicu, A.; Biraben, A.; Wendling, F.; Senhadji, L.; Merlet, I. Removal of muscle artifact from EEG data: Comparison between stochastic (ICA and CCA) and deterministic (EMD and wavelet-based) approaches. EURASIP J. Adv. Signal Process. 2012, 2012, 1–15. [Google Scholar] [CrossRef] [Green Version]
  48. Srivastava, G.; Crottaz-Herbette, S.; Lau, K.; Glover, G.H.; Menon, V. ICA-based procedures for removing ballistocardiogram artifacts from EEG data acquired in the MRI scanner. Neuroimage 2005, 24, 50–60. [Google Scholar] [CrossRef] [PubMed]
  49. Zhou, W.; Gotman, J. Removal of EMG and ECG artifacts from EEG based on wavelet transform and ICA. In Proceedings of the 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Francisco, CA, USA, 1–5 September 2004; pp. 392–395. [Google Scholar]
  50. Sedgwick, P. Limits of agreement (Bland-Altman method). Br. Med. J. 2013, 346, f1630. [Google Scholar] [CrossRef]
Figure 1. Schematic diagram of the proposed measuring systems (adapted from [40]).
Figure 1. Schematic diagram of the proposed measuring systems (adapted from [40]).
Applsci 11 04255 g001
Figure 2. Video recordings from two participants using Nikon 3500 digital camera.
Figure 2. Video recordings from two participants using Nikon 3500 digital camera.
Applsci 11 04255 g002
Figure 3. Overall system design of the non-contact SpO2 prediction.
Figure 3. Overall system design of the non-contact SpO2 prediction.
Applsci 11 04255 g003
Figure 4. Skin detection. (a) LAB colour space, (b) A channel of the LAB colour space, (c) B channel of the LAB colour space, (d) binary image of the A channel, (e) binary image of the B channel, (f) multiplied image of binary images.
Figure 4. Skin detection. (a) LAB colour space, (b) A channel of the LAB colour space, (c) B channel of the LAB colour space, (d) binary image of the A channel, (e) binary image of the B channel, (f) multiplied image of binary images.
Applsci 11 04255 g004
Figure 5. Average pixel signals for the red and green channels.
Figure 5. Average pixel signals for the red and green channels.
Applsci 11 04255 g005
Figure 6. Signal decomposition of i R   ( t ) using a complete EEMD technique with 100 realisations and 100 iterations with 0.2 noise standard deviation.
Figure 6. Signal decomposition of i R   ( t ) using a complete EEMD technique with 100 realisations and 100 iterations with 0.2 noise standard deviation.
Applsci 11 04255 g006
Figure 7. Signal decomposition of G   ( t ) using a complete EEMD technique with 100 realisations and 100 iterations with 0.2 of noise standard deviation.
Figure 7. Signal decomposition of G   ( t ) using a complete EEMD technique with 100 realisations and 100 iterations with 0.2 of noise standard deviation.
Applsci 11 04255 g007
Figure 8. Basic schematic diagram of the ICA technique.
Figure 8. Basic schematic diagram of the ICA technique.
Applsci 11 04255 g008
Figure 9. Signals based on the ICA technique.
Figure 9. Signals based on the ICA technique.
Applsci 11 04255 g009
Figure 10. Spectrum analysis of the selected signals and their spectrogram for (a) u 1 ( t ) signal, (b) u 2 ( t ) signal.
Figure 10. Spectrum analysis of the selected signals and their spectrogram for (a) u 1 ( t ) signal, (b) u 2 ( t ) signal.
Applsci 11 04255 g010
Figure 11. Correlation plot between the obtained SpO2 and the measured SpO2.
Figure 11. Correlation plot between the obtained SpO2 and the measured SpO2.
Applsci 11 04255 g011
Figure 12. The Bland–Altman and error ratios between the obtained SpO2 and the measured SpO2.
Figure 12. The Bland–Altman and error ratios between the obtained SpO2 and the measured SpO2.
Applsci 11 04255 g012
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Al-Naji, A.; Khalid, G.A.; Mahdi, J.F.; Chahl, J. Non-Contact SpO2 Prediction System Based on a Digital Camera. Appl. Sci. 2021, 11, 4255. https://doi.org/10.3390/app11094255

AMA Style

Al-Naji A, Khalid GA, Mahdi JF, Chahl J. Non-Contact SpO2 Prediction System Based on a Digital Camera. Applied Sciences. 2021; 11(9):4255. https://doi.org/10.3390/app11094255

Chicago/Turabian Style

Al-Naji, Ali, Ghaidaa A. Khalid, Jinan F. Mahdi, and Javaan Chahl. 2021. "Non-Contact SpO2 Prediction System Based on a Digital Camera" Applied Sciences 11, no. 9: 4255. https://doi.org/10.3390/app11094255

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop