skip to main content
10.1145/3313808.3313813acmconferencesArticle/Chapter ViewAbstractPublication PagesveeConference Proceedingsconference-collections
research-article

vSocket: virtual socket interface for RDMA in public clouds

Authors Info & Claims
Published:14 April 2019Publication History

ABSTRACT

RDMA has been widely adopted as a promising solution for high performance networks, but is still unavailable for a large number of socket-based applications running in public clouds due to the following reasons. There is no available virtualization technique of RDMA that can meet the cloud's requirements. Moreover, it is cost prohibitive to rewrite the socket-based applications with the Verbs API. To address the above problems, we present vSocket, a software-based RDMA virtualization framework for socket-based applications in public clouds. vSocket takes into account the demands of clouds such as security rules and network isolation, so it can be deployed in the current public clouds. Furthermore, vSocket provides native socket API so that socket-based applications can use it without any modifications. Finally, to validate the performance gains, we implemented a prototype and compared it with current virtual network solutions against 1) basic network benchmarks and 2) the Redis, a typical I/O intensive application. Experimental results show that the latency of basic benchmarks can be reduced by 88% and the throughput of Redis is improved by 4 times.

References

  1. 2018. Cloudius Systems. Seastar. http://seastar.io/ [Online; accessed 8-December-2018].Google ScholarGoogle Scholar
  2. 2018. Mellanox ConnectX-4 adaptor. http://www.mellanox.com/page/products_dyn?product_family=201&mtag=connectx_4_vpi_card. [Online; accessed 19-September-2018].Google ScholarGoogle Scholar
  3. 2018. Netperf benchmark. https://hewle_packard.github.io/netperf/ [Online; accessed 19-September-2018].Google ScholarGoogle Scholar
  4. 2018. Network ACLs. https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html. [Online; accessed 26-February-2019].Google ScholarGoogle Scholar
  5. 2018. The official benchmark of Redis. https://redis.io/topics/benchmarks [Online; accessed 19-September-2018].Google ScholarGoogle Scholar
  6. 2018. Perf Wiki. https://perf.wiki.kernel.org/index.php/Main_Page. [Online; accessed 19-September-2018].Google ScholarGoogle Scholar
  7. 2018. The RDMA Verbs Specification. http://www.rdmaconsortium.org/home/draft-hilland-iwarp-verbs-v1.0-RDMAC.pdf. [Online; accessed 19-September-2018].Google ScholarGoogle Scholar
  8. 2018. Redis, an open source (BSD licensed), in-memory data structure store. https://redis.io/ [Online; accessed 19-September-2018].Google ScholarGoogle Scholar
  9. 2018. Security Groups for Your VPC. https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html. [Online; accessed 8-December-2018].Google ScholarGoogle Scholar
  10. 2018. Single root I/O virtualization. http://pcisig.com/specifications/iov/single_root/ [Online; accessed 8-December-2018].Google ScholarGoogle Scholar
  11. Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). USENIX Association, Savannah, GA, 265-283. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Inho Cho, Keon Jang, and Dongsu Han. 2017. Credit-Scheduled Delay-Bounded Congestion Control for Datacenters. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '17). ACM, New York, NY, USA, 239-252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Aleksandar Dragojevic, Dushyanth Narayanan, Orion Hodson, and Miguel Castro. 2014. FaRM: Fast Remote Memory. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI'14). USENIX Association, Berkeley, CA, USA, 401-414. http://dl.acm.org/citation.cfm?id=2616448.2616486 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Adam Dunkels. 2001. Design and Implementation of the lwIP TCP/IP Stack. In Swedish Institute of Computer Science.Google ScholarGoogle Scholar
  15. M. Ferdman, P. Lotfi-Kamran, K. Balet, and B. Falsafi. 2011. Cuckoo directory: A scalable directory for many-core systems. In 2011 IEEE 17th International Symposium on High Performance Computer Architecture. 169-180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Daniel Firestone, Andrew Putnam, Sambhrama Mundkur, Derek Chiou, Alireza Dabagh, Mike Andrewartha, Hari Angepat, Vivek Bhanu, Adrian Caulfield, Eric Chung, et al. 2018. Azure Accelerated Networking: SmartNICs in the Public Cloud. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), Renton, WA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. D. Goldenberg, M. Kagan, R. Ravid, and M. S. Tsirkin. 2005. Zero copy sockets direct protocol over infiniband-preliminary implementation and performance analysis. In 13th Symposium on High Performance Interconnects (HOTI'05). 128-137. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Stefan Hajnoczi. 2011. QEMU Internals: vhost architecture. http://blog.vmsplice.net/2011/09/qemu-internals-vhost-architecture.html. [Online].Google ScholarGoogle Scholar
  19. Mark Handley, Costin Raiciu, Alexandru Agache, Andrei Voinescu, Andrew W. Moore, Gianni Antichi, and Marcin Wojcik. 2017. Rearchitecting Datacenter Networks and Stacks for Low Latency and High Performance. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '17). ACM, New York, NY, USA, 29-42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. SeanHefty. 2012. Rsockets. In 2012 OpenFabris International Workshop, Monterey, CA, USA.Google ScholarGoogle Scholar
  21. Jinho Hwang, K K Ramakrishnan, and Timothy Wood. 2015. NetVM: high performance and fiexible networking using virtualization on commodity platforms. IEEE Transactions on Network and Service Management 12, 1 (2015), 34-47.Google ScholarGoogle ScholarCross RefCross Ref
  22. DPDK Intel. 2018. Data plane development kit.Google ScholarGoogle Scholar
  23. Manish Jain, Ravi S Prasad, and Constantinos Dovrolis. 2003. The TCP bandwidth-delay product revisited: network buffering, cross traffic, and socket buffer auto-sizing. Technical Report. Georgia Institute of Technology.Google ScholarGoogle Scholar
  24. EunYoung Jeong, Shinae Woo, Muhammad Asim Jamshed, Haewon Jeong, Sunghwan Ihm, Dongsu Han, and KyoungSoo Park. 2014. mTCP: a Highly Scalable User-level TCP Stack for Multicore Systems. In NSDI, Vol. 14. 489-502. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Anuj Kalia, Michael Kaminsky, and David G. Andersen. 2014. Using RDMA Efficiently for Key-value Services. In Proceedings of the 2014 ACM Conference on SIGCOMM (SIGCOMM '14). ACM, New York, NY, USA, 295-306. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Patrick Kelsey. [n. d.]. Libuinet. https://github.com/pkelsey/libuinet [Online].Google ScholarGoogle Scholar
  27. M. Mahalingam, K. Duda, P. Agarwal, L. Kreeger, T. Sridhar, M. Bursell, and C. Wright. 2018. Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks. https://www.rfc-editor.org/info/rfc7348 [Online].Google ScholarGoogle Scholar
  28. Mellanox. [n. d.]. Mellanox Messaging Accelerator. http://www.mellanox.com/page/so_ware_vma [Online].Google ScholarGoogle Scholar
  29. Ben Pfaff, Justin Pettit, Teemu Koponen, Ethan J Jackson, Andy Zhou, Jarno Rajahalme, Jesse Gross, Alex Wang, Joe Stringer, Pravin Shelar, et al. 2015. The Design and Implementation of Open v Switch. In NSDI, Vol. 15. 117-130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Jonas Pfefferle et al. 2015. A Hybrid I/O Virtualization Framework for RDMA-capable Network Interfaces. In Proceedings of the VEE (VEE '15). 17-30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Adit Ranadive et al. 2012. Toward a Paravirtual v RDMA Device for VMware ESXi Guests. VMware Technical Journal. Georgia Institute of Technology, VMWare Inc.Google ScholarGoogle Scholar
  32. Rusty Russell. 2008. virtio: towards a de-facto standard for virtual I/O devices. ACM SIGOPS Operating Systems Review 42, 5 (2008), 95-103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Michael Scharf and Sebastian Kiesel. 2006. Head-of-line Blocking in TCP and SCTP: Analysis and Measurements. In GLOBECOM, Vol. 6. 1-5.Google ScholarGoogle ScholarCross RefCross Ref
  34. P. Shivam, P. Wyckoff, and D. Panda. 2001. EMP: Zero-Copy OS-Bypass NIC-Driven Gigabit Ethernet Message Passing. In SC '01: Proceedings of the 2001 ACM/IEEE Conference on Supercomputing. 49-49. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Jia Song and Jim Alves-Foss. 2012. Performance review of zero copy techniques. International Journal of Computer Science and Security (IJCSS) 6, 4 (2012), 256.Google ScholarGoogle Scholar
  36. Wikipedia. 2018. Token bucket. https://en.wikipedia.org/wiki/Token_bucket [Online; accessed 8-December-2018].Google ScholarGoogle Scholar
  37. Wikipedia. 2018. Traffic shaping. https://en.wikipedia.org/wiki/Traffic_shaping [Online; accessed 8-December-2018].Google ScholarGoogle Scholar
  38. Bairen Yi, Jiacheng Xia, Li Chen, and Kai Chen. 2017. Towards Zero Copy Dataflows Using RDMA. In Proceedings of the SIGCOMM Posters and Demos (SIGCOMM Posters and Demos '17). ACM, New York, NY, USA, 28-30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Tianlong Yu, Shadi Abdollahian Noghabi, Shachar Raindel, Hongqiang Liu, Jitu Padhye, and Vyas Sekar. 2016. FreeFlow: High Performance Container Networking. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (HotNets '16). ACM, New York, NY, USA, 43-49. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yibo Zhu, Haggai Eran, Daniel Firestone, Chuanxiong Guo, Marina Lipshteyn, Yehonatan Liron, Jitendra Padhye, Shachar Raindel, Mohamad Haj Yahia, and Ming Zhang. 2015. Congestion Control for Large-Scale RDMA Deployments. SIGCOMM Comput. Commun. Rev. 45, 4 (Aug. 2015), 523-536. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. vSocket: virtual socket interface for RDMA in public clouds

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      VEE 2019: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments
      April 2019
      206 pages
      ISBN:9781450360203
      DOI:10.1145/3313808
      • General Chair:
      • Jennifer Sartor,
      • Program Chairs:
      • Mayur Naik,
      • Chris Rossbach

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 April 2019

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate80of235submissions,34%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader