Liming Zhai
Yang Liu
ABSTRACT
Face de-identification (De-ID) removes face identity information in face images to avoid personal privacy leakage. Existing face De-ID breaks the raw identity by cutting out the face regions and recovering the corrupted regions via deep generators, which inevitably affect the generation quality and cannot control generation results according to subsequent intelligent tasks (eg., facial expression recognition). In this work, for the first attempt, we think the face De-ID from the perspective of attribute editing and propose an attribute-aware anonymization network (A3GAN) by formulating face De-ID as a joint task of semantic suppression and controllable attribute injection. Intuitively, the semantic suppression removes the identity-sensitive information in embeddings while the controllable attribute injection automatically edits the raw face along the attributes that benefit De-ID. To this end, we first design a multi-scale semantic suppression network with a novel suppressive convolution unit (SCU), which can remove the face identity along multi-level deep features progressively. Then, we propose an attribute-aware injective network (AINet) that can generate De-ID-sensitive attributes in a controllable way (i.e., specifying which attributes can be changed and which cannot) and inject them into the latent code of the raw face. Moreover, to enable effective training, we design a new anonymization loss to let the injected attributes shift far away from the original ones. We perform comprehensive experiments on four datasets covering four different intelligent tasks including face verification, face detection, facial expression recognition, and fatigue detection, all of which demonstrate the superiority of our face De-ID over state-of-the-art methods.
Supplemental Material
Available for Download
- Bhakti Baheti, Suhas Gajre, and Sanjay Talbar. 2018. Detection of distracted driver using convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 1032--1038.Google Scholar
Cross Ref
- Saheb Chhabra, Richa Singh, Mayank Vatsa, and Gaurav Gupta. 2018. Anonymizing k-facial attributes via adversarial perturbations. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 656--662.Google ScholarDigital Library
- Wenqing Chu, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Rongrong Ji. 2020. SSCGAN: Facial attribute editing via style skip connections. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XV 16. 414--429.Google Scholar
- Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4690--4699.Google ScholarCross Ref
- Julia Dietlmeier, Joseph Antony, Kevin McGuinness, and Noel E O'Connor. 2021. How important are faces for person re-identification?. In International Conference on Pattern Recognition (ICPR). IEEE, 6912--6919.Google ScholarCross Ref
- Omar Elharrouss, Noor Almaadeed, Somaya Al-Maadeed, and Younes Akbari. 2020. Image inpainting: A review. Neural Processing Letters 51, 2 (2020), 2007-- 2028.Google ScholarDigital Library
- Oran Gafni, Lior Wolf, and Yaniv Taigman. 2019. Live face de-identification in video. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9378--9387.Google ScholarCross Ref
- Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. Improved training of Wasserstein GANs. arXiv:arXiv preprint arXiv:1704.00028Google Scholar
- Zhenliang He, Wangmeng Zuo, Meina Kan, Shiguang Shan, and Xilin Chen. 2019. AttGAN: Facial attribute editing by only changing what you want. IEEE transactions on image processing 28, 11 (2019), 5464--5478.Google Scholar
- Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017).Google Scholar
- Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, Jilin Li, and Feiyue Huang. 2020. CurricularFace: adaptive curriculum learning loss for deep face recognition. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5901--5910.Google ScholarCross Ref
- Håkon Hukkelås, Rudolf Mester, and Frank Lindseth. 2019. DeepPrivacy: A generative adversarial network for face anonymization. In International Symposium on Visual Computing. 565--578.Google ScholarDigital Library
- Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. 448--456.Google ScholarDigital Library
- Amin Jourabloo, Xi Yin, and Xiaoming Liu. 2015. Attribute preserved face deidentification. In 2015 International conference on biometrics (ICB). 278--285.Google ScholarCross Ref
- Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4401--4410.Google ScholarCross Ref
- Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.Google Scholar
- Jeong-gi Kwak, David K Han, and Hanseok Ko. 2020. CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XIV 16. Springer, 524--540.Google Scholar
- Jian Li, YabiaoWang, ChanganWang, Ying Tai, Jianjun Qian, Jian Yang, Chengjie Wang, Jilin Li, and Feiyue Huang. 2019. DSFD: dual shot face detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5060--5069.Google ScholarCross Ref
- Yongxiang Li, Qianwen Lu, Qingchuan Tao, Xingbo Zhao, and Yanmei Yu. 2021. SF-GAN: Face De-identification Method without Losing Facial Attribute Information. IEEE Signal Processing Letters (2021).Google Scholar
- Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, and Shilei Wen. 2019. STGAN: A unified selective transfer network for arbitrary image attribute editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3673--3682.Google ScholarCross Ref
- Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision. 3730--3738.Google ScholarDigital Library
- Maxim Maximov, Ismail Elezi, and Laura Leal-Taixé. 2020. CIAGAN: Conditional identity anonymization generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5447--5456.Google ScholarCross Ref
- Blaz Meden, Peter Rot, Philipp Terhörst, Naser Damer, Arjan Kuijper, Walter J Scheirer, Arun Ross, Peter Peer, and Vitomir ?truc. 2021. Privacy--Enhancing Face Biometrics: A Comprehensive Survey. IEEE Transactions on Information Forensics and Security (2021).Google Scholar
- Vahid Mirjalili, Sebastian Raschka, and Arun Ross. 2020. PrivacyNet: semiadversarial networks for multi-attribute face privacy. IEEE Transactions on Image Processing 29 (2020), 9400--9412.Google ScholarCross Ref
- Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012. Noreference image quality assessment in the spatial domain. IEEE Transactions on image processing 21, 12 (2012), 4695--4708.Google ScholarDigital Library
- Mahyar Najibi, Pouya Samangouei, Rama Chellappa, and Larry S Davis. 2017. SSH: Single stage headless face detector. In Proceedings of the IEEE international conference on computer vision. 4875--4884.Google ScholarCross Ref
- Carman Neustaedter, Saul Greenberg, and Michael Boyle. 2006. Blur filtration fails to preserve privacy for home-based video conferencing. ACM Transactions on Computer-Human Interaction (TOCHI) 13, 1 (2006), 1--36.Google ScholarDigital Library
- Elaine M Newton, Latanya Sweeney, and Bradley Malin. 2005. Preserving privacy by de-identifying face images. IEEE transactions on Knowledge and Data Engineering 17, 2 (2005), 232--243.Google ScholarDigital Library
- José Ramón Padilla-López, Alexandros Andre Chaaraoui, and Francisco Flórez- Revuelta. 2015. Visual privacy protection methods: A survey. Expert Systems with Applications 42, 9 (2015), 4177--4195.Google ScholarDigital Library
- Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).Google Scholar
- Guim Perarnau, Joost Van De Weijer, Bogdan Raducanu, and Jose M Álvarez. 2016. Invertible conditional GANs for image editing. arXiv:arXiv preprint arXiv:1611.06355Google Scholar
- Slobodan Ribaric, Aladdin Ariyaeeinia, and Nikola Pavesic. 2016. Deidentification for privacy protection in multimedia content: A survey. Signal Processing: Image Communication 47 (2016), 131--151.Google ScholarDigital Library
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. 234--241.Google ScholarCross Ref
- Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 815--823.Google ScholarCross Ref
- SecureTrust. 2021. Data Privacy. https://www.securetrust.com/data-privacy/. Accessed: 2021-08--31.Google Scholar
- Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. 2020. Interpreting the latent space of GANs for semantic face editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9243--9252.Google ScholarCross Ref
- Qianru Sun, Liqian Ma, Seong Joon Oh, Luc Van Gool, Bernt Schiele, and Mario Fritz. 2018. Natural and effective obfuscation by head inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5050--5059.Google ScholarCross Ref
- Qianru Sun, Ayush Tewari, Weipeng Xu, Mario Fritz, Christian Theobalt, and Bernt Schiele. 2018. A hybrid model for identity obfuscation by face replacement. In Proceedings of the European Conference on Computer Vision. 553--569.Google ScholarCross Ref
- Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv:arXiv preprint arXiv:1607.08022Google Scholar
- Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao. 2020. Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6897--6906.Google ScholarCross Ref
- Ching-Hua Weng, Ying-Hsiu Lai, and Shang-Hong Lai. 2016. Driver drowsiness detection via a hierarchical temporal deep belief network. In Asian Conference on Computer Vision. 117--133.Google Scholar
- Guoxing Yang, Nanyi Fei, Mingyu Ding, Guangzhen Liu, Zhiwu Lu, and Tao Xiang. 2021. L2M-GAN: Learning To Manipulate Latent Space Semantics for Facial Attribute Editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2951--2960.Google ScholarCross Ref
- Shuo Yang, Ping Luo, Chen-Change Loy, and Xiaoou Tang. 2016. WIDER FACE: A face detection benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5525--5533.Google ScholarCross Ref
- Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2019. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4471--4480.Google ScholarCross Ref
- Gang Zhang, Meina Kan, Shiguang Shan, and Xilin Chen. 2018. Generative adversarial network with spatial attention for face attribute editing. In Proceedings of the European conference on computer vision (ECCV). 417--432.Google ScholarDigital Library
- Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2018. From facial expression recognition to interpersonal relation prediction. International Journal of Computer Vision 126, 5 (2018), 550--569.Google ScholarDigital Library
- Xin Zheng, Yanqing Guo, Huaibo Huang, Yi Li, and Ran He. 2020. A survey of deep facial attribute analysis. International Journal of Computer Vision 128, 8 (2020), 2002--2034.Google ScholarDigital Library
Index Terms
- A3GAN: Attribute-Aware Anonymization Networks for Face De-identification
Recommendations
Identity-Preserving Face Anonymization via Adaptively Facial Attributes Obfuscation
MM '21: Proceedings of the 29th ACM International Conference on MultimediaWith the popularity of using computer vision technology in monitoring system, there is an increasing societal concern on intruding people's privacy as the captured images/videos may contain identity-related information e.g. people's face. Existing ...
Collaborative Learning Network for Face Attribute Prediction
Computer Vision – ACCV 2016AbstractThis paper proposes a facial attributes learning algorithm with deep convolutional neural networks (CNN). Instead of jointly predicting all the facial attributes (40 attributes in our case) with a shared CNN feature extraction hierarchy, we ...
Lighting-aware face frontalization for unconstrained face recognition
Provide both lighting-recovered and lighting-normalized frontalized images.Basic frontalization with a generic 3D face model by the alignment of only five landmarks.Lighting recovered and normalized image filling by the symmetry of quotient image.LRFF ...
Comments