Deevashwer Rathee
ABSTRACT
We present CrypTFlow2, a cryptographic framework for secure inference over realistic Deep Neural Networks (DNNs) using secure 2-party computation. CrypTFlow2 protocols are both correct -- i.e., their outputs are bitwise equivalent to the cleartext execution -- and efficient -- they outperform the state-of-the-art protocols in both latency and scale. At the core of CrypTFlow2, we have new 2PC protocols for secure comparison and division, designed carefully to balance round and communication complexity for secure inference tasks. Using CrypTFlow2, we present the first secure inference over ImageNet-scale DNNs like ResNet50 and DenseNet121. These DNNs are at least an order of magnitude larger than those considered in the prior work of 2-party DNN inference. Even on the benchmarks considered by prior work, CrypTFlow2 requires an order of magnitude less communication and 20x-30x less time than the state-of-the-art.
Supplemental Material
- Mart'i n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Gregory S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian J. Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jó zefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané , Rajat Monga, Sherry Moore, Derek Gordon Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul A. Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda B. Vié gas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. CoRR , Vol. abs/1603.04467 (2016). https://arxiv.org/abs/1603.04467Google Scholar
- Nitin Agrawal, Ali Shahin Shamsabadi, Matt J. Kusner, and Adrià Gascó n. 2019. QUOTIENT: Two-Party Secure Neural Network Training and Prediction. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK, November 11--15, 2019. 1231--1247.Google ScholarDigital Library
- Gilad Asharov, Yehuda Lindell, Thomas Schneider, and Michael Zohner. 2013. More efficient oblivious transfer and extensions for faster secure computation. In 2013 ACM SIGSAC Conference on Computer and Communications Security, CCS'13, Berlin, Germany, November 4--8, 2013 , , Ahmad-Reza Sadeghi, Virgil D. Gligor, and Moti Yung (Eds.). ACM, 535--548. https://doi.org/10.1145/2508859.2516738Google ScholarDigital Library
- Marshall Ball, Brent Carmer, Tal Malkin, Mike Rosulek, and Nichole Schimanski. 2019. Garbled Neural Networks are Practical. IACR Cryptology ePrint Archive , Vol. 2019 (2019), 338. https://eprint.iacr.org/2019/338Google Scholar
- Assi Barak, Daniel Escudero, Anders Dalskov, and Marcel Keller. 2019. Secure Evaluation of Quantized Neural Networks. Cryptology ePrint Archive, Report 2019/131. https://eprint.iacr.org/2019/131.Google Scholar
- Donald Beaver. 1991. Efficient Multiparty Protocols Using Circuit Randomization. In Advances in Cryptology - CRYPTO '91, 11th Annual International Cryptology Conference, Santa Barbara, California, USA, August 11--15, 1991, Proceedings . 420--432.Google ScholarCross Ref
- Donald Beaver. 1996. Correlated Pseudorandomness and the Complexity of Private Computations. In Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing, Philadelphia, Pennsylvania, USA, May 22--24, 1996 , , Gary L. Miller (Ed.). ACM, 479--488. https://doi.org/10.1145/237814.237996Google ScholarDigital Library
- Mihir Bellare, Viet Tung Hoang, Sriram Keelveedhi, and Phillip Rogaway. 2013. Efficient Garbling from a Fixed-Key Blockcipher. In 2013 IEEE Symposium on Security and Privacy, SP 2013, Berkeley, CA, USA, May 19--22, 2013. IEEE Computer Society, 478--492. https://doi.org/10.1109/SP.2013.39Google ScholarDigital Library
- G. R. Blakley. 1979. Safeguarding cryptographic keys. In Managing Requirements Knowledge, International Workshop on. IEEE Computer Society, Los Alamitos, CA, USA, 313. https://doi.org/10.1109/AFIPS.1979.98Google Scholar
- Fabian Boemer, Anamaria Costache, Rosario Cammarota, and Casimir Wierzynski. 2019 a. nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data. In Proceedings of the 7th ACM Workshop on Encrypted Computing & Applied Homomorphic Cryptography, WAHC@CCS 2019, London, UK, November 11--15, 2019, Michael Brenner, Tancrè de Lepoint, and Kurt Rohloff (Eds.). ACM, 45--56. https://doi.org/10.1145/3338469.3358944Google ScholarDigital Library
- Fabian Boemer, Yixing Lao, Rosario Cammarota, and Casimir Wierzynski. 2019 b. nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data. In Proceedings of the 16th ACM International Conference on Computing Frontiers, CF 2019, Alghero, Italy, April 30 - May 2, 2019. 3--13.Google ScholarDigital Library
- Raphael Bost, Raluca Ada Popa, Stephen Tu, and Shafi Goldwasser. 2015. Machine Learning Classification over Encrypted Data. In 22nd Annual Network and Distributed System Security Symposium, NDSS 2015, San Diego, California, USA, February 8--11, 2015. The Internet Society. https://www.ndss-symposium.org/ndss2015/machine-learning-classification-over-encrypted-dataGoogle ScholarCross Ref
- Zvika Brakerski. 2012. Fully Homomorphic Encryption without Modulus Switching from Classical GapSVP. In Advances in Cryptology - CRYPTO 2012 - 32nd Annual Cryptology Conference, Santa Barbara, CA, USA, August 19--23, 2012. Proceedings (Lecture Notes in Computer Science, Vol. 7417), Reihaneh Safavi-Naini and Ran Canetti (Eds.). Springer, 868--886. https://doi.org/10.1007/978--3--642--32009--5_50Google Scholar
- Gilles Brassard, Claude Cré peau, and Jean-Marc Robert. 1986. All-or-Nothing Disclosure of Secrets. In Advances in Cryptology - CRYPTO '86, Santa Barbara, California, USA, 1986, Proceedings (Lecture Notes in Computer Science, Vol. 263), , Andrew M. Odlyzko (Ed.). Springer, 234--238. https://doi.org/10.1007/3--540--47721--7_17Google Scholar
- Niklas Bü scher, Daniel Demmler, Stefan Katzenbeisser, David Kretzmer, and Thomas Schneider. 2018. HyCC: Compilation of Hybrid Protocols for Practical Secure Computation. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15--19, 2018 , , David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.). ACM, 847--861. https://doi.org/10.1145/3243734.3243786Google Scholar
- Ran Canetti. 2000. Security and Composition of Multiparty Cryptographic Protocols. J. Cryptology , Vol. 13, 1 (2000), 143--202.Google ScholarDigital Library
- Nishanth Chandran, Divya Gupta, Aseem Rastogi, Rahul Sharma, and Shardul Tripathi. 2019. EzPC: Programmable and Efficient Secure Two-Party Computation for Machine Learning. In IEEE European Symposium on Security and Privacy, EuroS&P 2019, Stockholm, Sweden, June 17--19, 2019 . 496--511.Google Scholar
- Valerie Chen, Valerio Pastro, and Mariana Raykova. 2019. Secure Computation for Machine Learning With SPDZ . CoRR , Vol. abs/1901.00329 (2019). arxiv: 1901.00329 http://arxiv.org/abs/1901.00329Google Scholar
- Geoffroy Couteau. 2018. New Protocols for Secure Equality Test and Comparison. In Applied Cryptography and Network Security - 16th International Conference, ACNS 2018, Leuven, Belgium, July 2--4, 2018, Proceedings (Lecture Notes in Computer Science, Vol. 10892), , Bart Preneel and Frederik Vercauteren (Eds.). Springer, 303--320. https://doi.org/10.1007/978--3--319--93387-0_16Google Scholar
- Roshan Dathathri, Olli Saarikivi, Hao Chen, Kristin Lauter, Saeed Maleki, Madan Musuvathi, and Todd Mytkowicz. 2019. CHET: An Optimizing Compiler for Fully-Homomorphic Neural-Network Inferencing. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, Phoenix, AZ, USA, June 22--26, 2019. 142--156.Google ScholarDigital Library
- Daniel Demmler, Thomas Schneider, and Michael Zohner. 2015. ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation. In 22nd Annual Network and Distributed System Security Symposium, NDSS 2015, San Diego, California, USA, February 8--11, 2015 .Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20--25 June 2009, Miami, Florida, USA. 248--255.Google ScholarCross Ref
- Ghada Dessouky, Farinaz Koushanfar, Ahmad-Reza Sadeghi, Thomas Schneider, Shaza Zeitouni, and Michael Zohner. 2017. Pushing the Communication Barrier in Secure Computation using Lookup Tables. In 24th Annual Network and Distributed System Security Symposium, NDSS 2017, San Diego, California, USA, February 26 - March 1, 2017. The Internet Society. https://www.ndss-symposium.org/ndss2017/ndss-2017-programme/pushing-communication-barrier-secure-computation-using-lookup-tables/Google ScholarCross Ref
- Daniel Escudero, Satrajit Ghosh, Marcel Keller, Rahul Rachuri, and Peter Scholl. 2020. Improved Primitives for MPC over Mixed Arithmetic-Binary Circuits. In Advances in Cryptology - CRYPTO 2020 - 40th Annual International Cryptology Conference .Google Scholar
- Shimon Even, Oded Goldreich, and Abraham Lempel. 1985. A Randomized Protocol for Signing Contracts. Commun. ACM , Vol. 28, 6 (1985), 637--647. https://doi.org/10.1145/3812.3818Google ScholarDigital Library
- Junfeng Fan and Frederik Vercauteren. 2012. Somewhat Practical Fully Homomorphic Encryption. Cryptology ePrint Archive, Report 2012/144. http://eprint.iacr.org/2012/144.Google Scholar
- Juan A. Garay, Berry Schoenmakers, and José Villegas. 2007. Practical and Secure Solutions for Integer Comparison. In Public Key Cryptography - PKC 2007, 10th International Conference on Practice and Theory in Public-Key Cryptography, Beijing, China, April 16--20, 2007, Proceedings (Lecture Notes in Computer Science, Vol. 4450), Tatsuaki Okamoto and Xiaoyun Wang (Eds.). Springer, 330--342. https://doi.org/10.1007/978--3--540--71677--8_22Google Scholar
- Craig Gentry. 2009. Fully homomorphic encryption using ideal lattices. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31 - June 2, 2009, Michael Mitzenmacher (Ed.). ACM, 169--178. https://doi.org/10.1145/1536414.1536440Google ScholarDigital Library
- Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin E. Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19--24, 2016 . 201--210.Google Scholar
- Oded Goldreich, Silvio Micali, and Avi Wigderson. 1987. How to Play any Mental Game or A Completeness Theorem for Protocols with Honest Majority. In Proceedings of the 19th Annual ACM Symposium on Theory of Computing, 1987, New York, New York, USA. 218--229.Google ScholarDigital Library
- Shay Gueron. 2016. AES-GCM-SIV implementations (128 and 256 bit) . https://github.com/Shay-Gueron/AES-GCM-SIV .Google Scholar
- Shay Gueron, Yehuda Lindell, Ariel Nof, and Benny Pinkas. 2018. Fast Garbling of Circuits Under Standard Assumptions. J. Cryptol. , Vol. 31, 3 (2018). https://doi.org/10.1007/s00145-017--9271-yGoogle ScholarDigital Library
- C. Guo, J. Katz, X. Wang, and Y. Yu. 2020. Efficient and Secure Multiparty Computation from Fixed-Key Block Ciphers. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, Los Alamitos, CA, USA, 247--263. https://doi.org/10.1109/SP.2020.00016Google ScholarCross Ref
- Carmit Hazay, Yuval Ishai, Antonio Marcedone, and Muthuramakrishnan Venkitasubramaniam. 2019. LevioSA: Lightweight Secure Arithmetic Computation. In Proceedings of the 2019 ACM Conference on Computer and Communications Security, CCS 2019, London, UK, November 11--15, 2019. 327--344.Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27--30, 2016 . 770--778.Google Scholar
- Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. 2017. Densely Connected Convolutional Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21--26, 2017 . 2261--2269.Google Scholar
- Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized Neural Networks. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5--10, 2016, Barcelona, Spain, Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.). 4107--4115.Google ScholarDigital Library
- Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Song Han, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and textless1MB model size. CoRR , Vol. abs/1602.07360 (2016). arxiv: 1602.07360 http://arxiv.org/abs/1602.07360Google Scholar
- Yuval Ishai, Joe Kilian, Kobbi Nissim, and Erez Petrank. 2003. Extending Oblivious Transfers Efficiently. In Advances in Cryptology - CRYPTO 2003, 23rd Annual International Cryptology Conference, Santa Barbara, California, USA, August 17--21, 2003, Proceedings (Lecture Notes in Computer Science, Vol. 2729), , Dan Boneh (Ed.). Springer, 145--161. https://doi.org/10.1007/978--3--540--45146--4_9Google Scholar
- Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew G. Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18--22, 2018. 2704--2713.Google Scholar
- Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. GAZELLE: A Low Latency Framework for Secure Neural Network Inference. In 27th USENIX Security Symposium, USENIX Security 2018, Baltimore, MD, USA, August 15--17, 2018 . 1651--1669.Google ScholarDigital Library
- Vladimir Kolesnikov and Ranjit Kumaresan. 2013. Improved OT Extension for Transferring Short Secrets. In Advances in Cryptology - CRYPTO 2013 - 33rd Annual Cryptology Conference, Santa Barbara, CA, USA, August 18--22, 2013. Proceedings, Part II (Lecture Notes in Computer Science, Vol. 8043), , Ran Canetti and Juan A. Garay (Eds.). Springer, 54--70. https://doi.org/10.1007/978--3--642--40084--1_4Google Scholar
- Nishant Kumar, Mayank Rathee, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. 2020. CrypTFlow: Secure TensorFlow Inference. In 2020 IEEE Symposium on Security and Privacy, S&P 2020, San Francisco, CA, USA, May 18--20, 2020 . 1521--1538.Google Scholar
- Kim Laine. 2017. Simple Encrypted Arithmetic Library 2.3.1 . https://www.microsoft.com/en-us/research/uploads/prod/2017/11/sealmanual-2--3--1.pdf .Google Scholar
- Yehuda Lindell. 2016. How To Simulate It - A Tutorial on the Simulation Proof Technique. Cryptology ePrint Archive, Report 2016/046. https://eprint.iacr.org/2016/046.Google Scholar
- Jian Liu, Mika Juuti, Yao Lu, and N. Asokan. 2017. Oblivious Neural Network Predictions via MiniONN Transformations. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017. 619--631.Google ScholarDigital Library
- Pratyush Mishra, Ryan Lehmkuhl, Akshayaram Srinivasan, Wenting Zheng, and Raluca Ada Popa. 2020. Delphi: A Cryptographic Inference Service for Neural Networks. In 29th USENIX Security Symposium, USENIX Security 20. Boston, MA.Google Scholar
- Payman Mohassel and Peter Rindal. 2018. ABY(^mbox3 ): A Mixed Protocol Framework for Machine Learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, October 15--19, 2018. 35--52.Google ScholarDigital Library
- Payman Mohassel and Yupeng Zhang. 2017. SecureML: A System for Scalable Privacy-Preserving Machine Learning. In 2017 IEEE Symposium on Security and Privacy, S&P 2017, San Jose, CA, USA, May 22--26, 2017 . 19--38.Google Scholar
- Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. 2019. Data-Free Quantization Through Weight Equalization and Bias Correction. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE , 1325--1334.Google Scholar
- Erman Pattuk, Murat Kantarcioglu, Huseyin Ulusoy, and Bradley A. Malin. 2016. CheapSMC: A Framework to Minimize Secure Multiparty Computation Cost in the Cloud. In Data and Applications Security and Privacy XXX - 30th Annual IFIP WG 11.3 Conference, DBSec 2016, Trento, Italy, July 18--20, 2016. Proceedings (Lecture Notes in Computer Science, Vol. 9766), Silvio Ranise and Vipin Swarup (Eds.). Springer, 285--294.Google Scholar
- Michael O. Rabin. 1981. How to exchange secrets with oblivious transfer. Technical Report TR-81, Aiken Computation Lab, Harvard University. https://eprint.iacr.org/2005/187.pdf.Google Scholar
- M. Sadegh Riazi, Mohammad Samragh, Hao Chen, Kim Laine, Kristin E. Lauter, and Farinaz Koushanfar. 2019. XONN: XNOR-based Oblivious Deep Neural Network Inference. In 28th USENIX Security Symposium, USENIX Security 2019, Santa Clara, CA, USA, August 14--16, 2019 . 1501--1518.Google Scholar
- M. Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M. Songhori, Thomas Schneider, and Farinaz Koushanfar. 2018. Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security, AsiaCCS 2018, Incheon, Republic of Korea, June 04-08, 2018 . 707--721. https://doi.org/10.1145/3196494.3196522Google ScholarDigital Library
- Bita Darvish Rouhani, M. Sadegh Riazi, and Farinaz Koushanfar. 2018. Deepsecure: scalable provably-secure deep learning. In Proceedings of the 55th Annual Design Automation Conference, DAC 2018, San Francisco, CA, USA, June 24--29, 2018. ACM, 2:1--2:6.Google ScholarDigital Library
- Adi Shamir. 1979. How to Share a Secret. Commun. ACM , Vol. 22, 11 (1979), 612--613. https://doi.org/10.1145/359168.359176Google ScholarDigital Library
- N.P. Smart and F. Vercauteren. 2011. Fully Homomorphic SIMD Operations. Cryptology ePrint Archive, Report 2011/133. http://eprint.iacr.org/2011/133.Google Scholar
- Sameer Wagh, Divya Gupta, and Nishanth Chandran. 2019. SecureNN: 3-Party Secure Computation for Neural Network Training . PoPETs , Vol. 2019, 3 (2019), 26--49.Google ScholarCross Ref
- Xiao Wang, Alex J. Malozemoff, and Jonathan Katz. 2016. EMP-toolkit: Efficient MultiParty computation toolkit . https://github.com/emp-toolkit .Google Scholar
- Andrew Chi-Chih Yao. 1986. How to Generate and Exchange Secrets (Extended Abstract). In 27th Annual Symposium on Foundations of Computer Science, Toronto, Canada, 27--29 October 1986 . IEEE Computer Society, 162--167. https://doi.org/10.1109/SFCS.1986.25Google ScholarDigital Library
- Wenting Zheng, Raluca Ada Popa, Joseph E. Gonzalez, and Ion Stoica. 2019. Helen: Maliciously Secure Coopetitive Learning for Linear Models. In 2019 IEEE Symposium on Security and Privacy, S&P 2019, San Francisco, CA, USA, May 19--23, 2019 . 724--738.Google Scholar
- Xiaoyong Zhu, George Iordanescu, Ilia Karmanov, and Mazen Zawaideh. 2018. https://blogs.technet.microsoft.com/machinelearning/2018/03/07/using-microsoft-ai-to-build-a-lung-disease-prediction-model-using-chest-x-ray-images/Google Scholar
Index Terms
- CrypTFlow2: Practical 2-Party Secure Inference
Recommendations
An Efficient Protocol for Secure Two-Party Computation in the Presence of Malicious Adversaries
We show an efficient secure two-party protocol, based on Yao's construction, which provides security against malicious adversaries. Yao's original protocol is only secure in the presence of semi-honest adversaries, and can be transformed into a protocol ...
On the Power of Secure Two-Party Computation
Proceedings, Part II, of the 36th Annual International Cryptology Conference on Advances in Cryptology --- CRYPTO 2016 - Volume 9815Ishai, Kushilevitz, Ostrovsky and Sahai STOC 2007, SIAM JoC 2009 introduced the powerful "MPC-in-the-head" technique that provided a general transformation of information-theoretic MPC protocols secure against passive adversaries to a ZK proof in a "...
Secure Two-Party Computation via Cut-and-Choose Oblivious Transfer
Protocols for secure two-party computation enable a pair of parties to compute a function of their inputs while preserving security properties such as privacy, correctness and independence of inputs. Recently, a number of protocols have been proposed ...
Comments