Skip to main content
SpringerLink
Log in

Protection of visual privacy in videos acquired with RGB cameras for active and assisted living applications

  • 1163: Large-scale multimedia signal processing for security and digital forensics
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Active and assisted living technologies are much needed, but some aspects of them cause user rejection due to concerns on privacy. This is even more concerning to users when visual information is used, processed, and transmitted. To respond to these concerns, and maximise user acceptance, visual privacy protection measures have to be put in place. In the past, human detection and object segmentation in video were constrained by technological limitations, and could only run with specific hardware and sensors. This paper introduces a proposal for an RGB-only based visual privacy preservation filter, which capitalises on ‘deep learning’-based segmentation and pose detectors. A background update scheme is presented, which limits leakage of sensitive information when detection fails. Dilation of the mask can further prevent information leakage, but a trade-off is necessary to correctly update background information. This is achieved via a specific study which is also presented. A comparative study is performed to determine the best configuration for privacy preservation. Results show that union of dilated masks from different deep networks achieves the best overall result.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. from: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32016R0679(Visited: Feb 2020)

  2. For more information visit: www.paal-project.eu(last access: Feb 2020)

  3. Both are available online on GitHub. The reader is referred to the references section for details.

  4. Available under request from: http://velastin.dynu.com/MuHAVi-MAS/(last access: Feb 2020)

  5. Available from: https://project.inria.fr/toyotasmarthome/(Last access: Feb 2020)

References

  1. Abdulla W (2017) Mask r-cnn for object detection and instance segmentation on keras and tensorflow. https://github.com/matterport/Mask_RCNN

  2. Arning K, Ziefle M (2015) “get that camera out of my house!” conjoint measurement of preferences for video-based healthcare monitoring systems in private and public places. In: Geissbühler A, Demongeot J, Mokhtari M, Abdulrazak B, Aloulou H (eds) Inclusive smart cities and e-health, pp 152–164. Springer International Publishing, Cham

  3. Babiceanu RF, Bojda P, Seker R, Alghumgham MA (2015) An onboard uas visual privacy guard system. In: 2015 Integrated communication, navigation and surveillance conference (ICNS), pp J1–1–J1–8

  4. Bouwmans T (2014) Traditional and recent approaches in background modeling for foreground detection: An overview. Comp Sci Rev 11-12:31–66. https://doi.org/10.1016/j.cosrev.2014.04.001

    Article  MATH  Google Scholar 

  5. Bouwmans T, Javed S, Sultana M, Jung SK (2019) Deep neural network concepts for background subtraction:a systematic review and comparative evaluation. Neural Netw 117:8–66. https://doi.org/10.1016/j.neunet.2019.04.024

    Article  Google Scholar 

  6. Calvaresi D, Cesarini D, Sernani P, Marinoni M, Dragoni AF, Sturm A (2017) Exploring the ambient assisted living domain: a systematic review. Journal of Ambient Intelligence and Humanized Computing 8(2):239–257

    Article  Google Scholar 

  7. Chaaraoui AA, Padilla-López JR, Ferrández-Pastor FJ, Nieto-Hidalgo M, Flórez-Revuelta F (2014) A vision-based system for intelligent monitoring: Human behaviour analysis and privacy by context. Sensors 14(5):8895–8925. https://doi.org/10.3390/s140508895

    Article  Google Scholar 

  8. Chen J, Konrad J, Ishwar P (2018) Vgan-based image representation learning for privacy-preserving facial expression recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1570–1579

  9. Çiftçi S, Akyüz AO, Ebrahimi T (2017) A reliable and reversible image privacy protection based on false colors. IEEE transactions on Multimedia 20(1):68–81

    Article  Google Scholar 

  10. Climent-Pérez P, Spinsante S, Mihailidis A, Florez-Revuelta F (2020) A review on video-based active and assisted living technologies for automated lifelogging. Expert Syst Appl 139:112847. https://doi.org/10.1016/j.eswa.2019.112847

    Article  Google Scholar 

  11. Dabrowski A, Krombholz K, Weippl ER, Echizen I (2015) Smart privacy visor: Bridging the privacy gap. In: Abramowicz W (ed) Business information systems workshops, pp 235–247. Springer International Publishing, Cham

  12. Das S, Dai R, Koperski M, Minciullo L, Garattoni L, Bremond F, Francesca G (2019) Toyota smarthome: Real-world activities of daily living. In: The IEEE international conference on computer vision (ICCV)

  13. Güler RA, Neverova N, Kokkinos I (2018) Densepose: Dense human pose estimation in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7297–7306

  14. Hasan R, Shaffer P, Crandall D, Apu Kapadia ET, et al. (2017) Cartooning for enhanced privacy in lifelogging and streaming videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 29–38

  15. He K, Gkioxari G, Dollar P, Girshick R (2017) Mask r-cnn. In: The IEEE international conference on computer vision (ICCV)

  16. Krombholz K, Dabrowski A, Smith M, Weippl E (2017) Exploring design directions for wearable privacy. In: USEC’17

  17. Nguyen T-H-C, Nebel J-C, Florez-Revuelta F (2016) Recognition of activities of daily living with egocentric vision: A review. Sensors 16(1):72. https://doi.org/10.3390/s16010072

    Article  Google Scholar 

  18. Offermann-van Heek J, Arning K, Ziefle M (2019) All eyes on you! impact of location, camera type, and privacy-security-trade-off on the acceptance of surveillance technologies. In: Donnellan B, Klein C, Helfert M, Gusikhin O, Pascoal A (eds) Smart cities, green technologies, and intelligent transport systems, pp 131–149. Springer International Publishing, Cham

  19. Offermann-van Heek J, Ziefle M (2018) They don’t care about us! care personnel’s perspectives on ambient assisted living technology usage: Scenario-based survey study. JMIR Rehabil Assist Technol 5(2):e10424. https://doi.org/10.2196/10424

    Article  Google Scholar 

  20. Orekondy T, Schiele B, Fritz M (2017) Towards a visual privacy advisor: Understanding and predicting privacy risks in images. In: Proceedings of the IEEE international conference on computer vision, pp 3686–3695

  21. Padilla-López J, Chaaraoui A, Gu F, Flórez-Revuelta F (2015) Visual privacy by context: proposal and evaluation of a level-based visualisation scheme. Sensors 15(6):12959–12982

    Article  Google Scholar 

  22. Planinc R, Chaaraoui AA, Kampel M, Florez-Revuelta F (2016) Computer vision for active and assisted living. In: Active and assisted living: technologies and applications, healthcare technologies, pp 57–79. Institution of Engineering and Technology

  23. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances in neural information processing systems 28, pp 91–99. Curran Associates, Inc.

  24. Ribaric S, Ariyaeeinia A, Pavesic N (2016) De-identification for privacy protection in multimedia content: A survey. Signal Process Image Commun 47:131–151

    Article  Google Scholar 

  25. Sathyanarayana S, Satzoda RK, Sathyanarayana S, Thambipillai S (2018) Vision-based patient monitoring: a comprehensive review of algorithms and technologies. Journal of Ambient Intelligence and Humanized Computing 9(2):225–251. https://doi.org/10.1007/s12652-015-0328-1

    Article  Google Scholar 

  26. Schiff J, Meingast M, Mulligan DK, Sastry S, Goldberg K (2009) Respectful cameras: Detecting visual markers in real-time to address privacy concerns. In: Senior A (ed) Protecting privacy in video surveillance, pp 65–89. Springer London, London

  27. Shu J, Zheng R, Hui P (2018) Cardea: Context-aware visual privacy protection for photo taking and sharing. In: Proceedings of the 9th ACM multimedia systems conference, MMSys ’18, pp 304–315. Association for Computing Machinery, New York, NY, USA

  28. Singh S, Velastin SA, Ragheb H (2010) Muhavi: A multicamera human action video dataset for the evaluation of action recognition methods. In: Advanced Video and Signal Based Surveillance (AVSS), 2010 Seventh IEEE International Conference on, pp 48–55. IEEE

  29. Sobral A (2013) BGSLibrary: An opencv c++ background subtraction library. In: IX Workshop de Visão Computacional (WVC’2013). Rio de Janeiro, Brazil. https://github.com/andrewssobral/bgslibrary

  30. Steil J, Koelle M, Heuten W, Boll S, Bulling A (2019) Privaceye: Privacy-preserving head-mounted eye tracking using egocentric scene image and eye movement features. In: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ETRA ’19. Association for Computing Machinery, New York, NY, USA

  31. Varol G, Romero J, Martin X, Mahmood N, Black MJ, Laptev I, Schmid C (2017) Learning from synthetic humans. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  32. Wang S, Cheung S-CS, Sajid H (2017) Visual bubble: Protecting privacy in wearable cameras. IEEE Consumer Electronics Magazine 7(1):95–105

    Article  Google Scholar 

  33. Wu Y, Kirillov A, Massa F, Lo W-Y, Girshick R (2019) Detectron2. https://github.com/facebookresearch/detectron2https://github.com/facebookresearch/detectron2

  34. Wu Z, Wang Z, Wang Z, Jin H (2018) Towards privacy-preserving visual recognition via adversarial training: A pilot study. In: Proceedings of the European conference on computer vision (ECCV), pp 606–624

Download references

Acknowledgements

This work is part of the PAAL – “Privacy-Aware and Acceptable Lifelogging services for older and frail people” project: The support of the Joint Programme Initiative “More Years, Better Lives” (JPI MYBL, award number: PAAL_JTC2017) and the Spanish Agencia Estatal de Investigación (grant no: PCIN-2017-114) is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department of Computing Technology, University of Alicante, P.O. Box 99, E-03080, Alicante, Spain

    Pau Climent-Pérez & Francisco Florez-Revuelta

Authors
  1. Pau Climent-Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco Florez-Revuelta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pau Climent-Pérez.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest. Additionally, the funding bodies had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Climent-Pérez, P., Florez-Revuelta, F. Protection of visual privacy in videos acquired with RGB cameras for active and assisted living applications. Multimed Tools Appl 80, 23649–23664 (2021). https://doi.org/10.1007/s11042-020-10249-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10249-1

Keywords

Search

Navigation