Skip to main content
SpringerLink
Log in

HONE: Joint Host-Network Traffic Management in Software-Defined Networks

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Applications running in modern datacenters interact with the underlying network in complex ways, making administrators run multiple traffic management tasks to tune the system. However, today’s traffic management solutions are limited by an artificial division between the hosts and the network. While network devices only have knowledge of the network layer, the hosts can provide more visibility into how applications interact with the network. This paper proposes to extend the scope of traffic management to the end-host network stack. We present a software-defined networking platform for joint HOst-NEtwork (HONE) traffic management. HONE presents a uniform view of a diverse collection of measurement data, minimizes measurement overhead by performing lazy materialization of fine-grained statistics, and scales the analysis by processing data locally on the end hosts. HONE offers a simple and expressive programming framework for network and service administrators. We evaluate HONE by implementing several canonical traffic management applications, measuring its efficiency with micro-benchmarks, and demonstrating its scalability with larger experiments on Amazon EC2.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. In the syntax, the star operator (*) glues together the various query components. Each query term generates a bit of abstract syntax that our runtime system interprets.

  2. The HONE controller ships the source code of the local portion of management tasks to the host agent. Since HONE programs are written in Python, the agent can execute them with its local Python interpreter, and thus avoids the difficulties of making the programs compatible with diverse environments on the hosts.

  3. The runtime uses information from the directory service to discover and organize hosts.

  4. One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0–1.2 GHz 2007 Opteron or Xeon processor.

References

  1. Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of datacenter traffic: measurements and analysis. In: ACM IMC (2009)

  2. Wu, H., Feng, Z., Guo, C., Zhang, Y.: ICTCP: Incast congestion control for TCP in data center networks. In: ACM CoNEXT (2010)

  3. Yu, M., Greenberg, A., Maltz, D., Rexford, J., Yuan, L., Kandula, S., Kim, C.: Profiling network performance for multi-tier data center applications. In: USENIX NSDI (2011)

  4. Al-Fares, M., Radhakrishnan, S., Raghavan, B., Huang, N., Vahdat, A.: Hedera: Dynamic flow scheduling for data center networks. In: USENIX NSDI (2010)

  5. Patel, P., Bansal, D., Yuan, L., Murthy, A., Greenberg, A., Maltz, D.A.., Kern, R., Kumar, H., Zikos, M., Wu, H., Kim, C., Karri, N.: Ananta: cloud scale load balancing. In: ACM SIGCOMM (2013)

  6. Raghavan, B., Vishwanath, K., Ramabhadran, S., Yocum, K., Snoeren, A.C.: Cloud control with distributed rate limiting. In: ACM SIGCOMM (2007)

  7. Jain, S., Kumar, A., Mandal, S., Ong, J., Poutievski, L., Singh, A., Venkata, S., Wanderer, J., Zhou, J., Zhu, M., Zolla, J., Hölzle, U., Stuart, S., Vahdat, A.: B4: Experience with a globally-deployed software defined WAN. In: ACM SIGCOMM (2013)

  8. Hong, C.-Y., Kandula, S., Mahajan, R., Zhang, M., Gill, V., Nanduri, M., Wattenhofer, R.: Achieving high utilization with software-driven WAN. In: ACM SIGCOMM (2013)

  9. Dixon, C., Uppal, H., Brajkovic, V., Brandon, D., Anderson, T., Krishnamurthy, A.: ETTM: A scalable fault tolerant network manager. In: USENIX NSDI (2011)

  10. Karagiannis, T., Mortier, R., Rowstron, A.: Network exception handlers: host-network control in enterprise networks. In: ACM SIGCOMM (2008)

  11. Sherry, J., Kim, D.C., Mahalingam, S.S., Tang, A., Wang, S., Ratnasamy, S.: Netcalls: End host function calls to network traffic processing services. Technical report UCB/EECS-2012-175, U.C. Berkeley (2012)

  12. Shieh, A., Kandula, S., Greenberg, A., Kim, C., Saha, B.: Sharing the data center network. In: USENIX NSDI (2011)

  13. Zats, D., Das, T., Mohan, P., Borthakur, D., Katz, R.: DeTail: Reducing the flow completion time tail in datacenter networks. In: ACM SIGCOMM (2012)

  14. Pfaff, B., Pettit, J., Amidon, K., Casado, M., Koponen, T., Shenker, S.: Extending networking into the virtualization layer. In: ACM HotNets (2009)

  15. Hong, C.-Y., Caesar, M., Godfrey, B.P.: Finishing flows quickly with preemptive scheduling. In: ACM SIGCOMM (2012)

  16. Wilson, C., Ballani, H., Karagiannis, T., Rowtron, A.: Better never than late: meeting deadlines in datacenter networks. In: ACM SIGCOMM (2011)

  17. Event Tracing for Windows. http://support.microsoft.com/kb/2593157 (2011)

  18. Web10G Project. http://web10g.org/ (2012)

  19. VMWare vCenter Suite. http://www.vmware.com/products/datacenter-virtualization/vcenter-operations-management/overview.html (2013)

  20. Linux Advanced Routing and Traffic Control. http://www.lartc.org/ (2000)

  21. Netfilter.org. http://www.netfilter.org/ (1999)

  22. Open vSwitch. http://openvswitch.org/ (2011)

  23. Mathis, M., Heffner, J., Raghunarayan, R.: TCP Extended Statistics MIB. RFC 4898 (2007)

  24. McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., Turner, J.: OpenFlow: enabling innovation in campus networks. ACM CCR 38(2), 69–74 (2008)

    Article  Google Scholar 

  25. Curtis, A., Kim, W., Yalagandula, P.: Mahout: low-overhead datacenter traffic management using end-host-based elephant detection. In: IEEE INFOCOM (2011)

  26. Popa, L., Kumar, G., Chowdhury, M., Krishnamurthy, A., Ratnasamy, S., Stoica, I.: FairCloud: sharing the network in cloud computing. In: ACM SIGCOMM (2012)

  27. Borders, K., Springer, J., Burnside, M.: Chimera: A declarative language for streaming network traffic analysis. In: USENIX Security (2012)

  28. Cranor, C., Johnson, T., Spataschek, O., Shkapenyuk, V.: Gigascope: A stream database for network applications. In: ACM SIGMOD (2003)

  29. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: USENIX OSDI (2004)

  30. Dean, J., Ghemawat, S.: MapReduce: A flexible data processing tool. Commun. ACM 53(1), 72–77 (2010). ISSN 0001–0782.2010. doi:10.1145/1629175.1629198

  31. Pavlo, A., Paulson, E., Rasin, A., Abadi, D.J., DeWitt, D.J., Madden, S., Stonebraker, M.: A comparison of approaches to large-scale data analysis. In: ACM SIGMOD (2009)

  32. NTP: The Network Time Protocol. http://www.ntp.org/ (2003)

  33. van Renesse, R., Bozdog, A.: Willow: DHT, aggregation, and publish/subscribe in one protocol. In: IPTPS (2004)

  34. Yalagandula, P., Dahlin, M.: A scalable distributed information management system. In: ACM SIGCOMM (2004)

  35. FloodLight. http://floodlight.openflowhub.org/ (2011)

  36. Al-Fares, M., Loukissas, A., Vahdat, A.: A scalable, commodity data center network architecture. In: ACM SIGCOMM (2008)

  37. Cooke, E., Mortier, R., Donnelly, A., Barham, P., Isaacs, R.: Reclaiming network-wide visibility using ubiquitous end system monitors. In: USENIX ATC (2006)

  38. Ghobadi, M., Yeganeh, S.H., Ganjali, Y.: Rethinking end-to-end congestion control in software-defined networks. In: ACM HotNets (2012)

  39. Lee, Y., Bernstein, G., So, N., Kim, T.Y., Shiomoto, K., de Dios, O.G.: Research proposal for cross stratum optimization, (CSO) between data centers and networks. http://tools.ietf.org/html/draft-lee-cross-stratum-optimization-datacenter-00 (2011)

  40. OpenTSDB Project. http://www.opentsdb.net/ (2010)

  41. Boundary. http://www.boundary.com/ (2010)

  42. Ou, X., Govindavajhala, S., Appel, A.W.: MulVAL: A logic-based network security analyzer. In: USENIX security (2005)

  43. Foster, N., Harrison, R., Freedman, M.J., Monsanto, C., Rexford, J., Story, A., Walker, D.: Frenetic: A network programming language. In: ACM ICFP (2011)

  44. Monsanto, C., Reich, J., Foster, N., Rexford, J., Walker, D.: Composing software-defined networks. In: USENIX NSDI (2013)

  45. Nelson, T., Guha, A., Dougherty, D.J., Fisler, K., Krishnamurthi, S.: A balance of power: expressive, analyzable controller programming. In: ACM SIGCOMM HotSDN (2013)

  46. Yuan, L., Chuah, C.-N., Mohapatra, P.: ProgME: towards programmable network measurement. In: ACM SIGCOMM (2007)

  47. Yu, M., Jose, L., Miao, R.: Software defined traffic measurement with OpenSketch. In: USENIX NSDI (2013)

Download references

Acknowledgments

This work is supported by grants NSF NETS 1162112 and DARPA MRC 2012-00310-02. It is also partially supported by Cisco, Google, Software R&D Center at Samsung Electronics, and USC Zumberge Research and innovation fund.

Author information

Authors and Affiliations

  1. Princeton University, 35 Olden Street, Princeton, NJ, 08540, USA

    Peng Sun, Michael J. Freedman, Jennifer Rexford & David Walker

  2. University of Southern California, 941 Bloom Walk, SAL 228, Los Angeles, CA, 90089, USA

    Minlan Yu

Authors
  1. Peng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minlan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael J. Freedman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jennifer Rexford
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Walker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, P., Yu, M., Freedman, M.J. et al. HONE: Joint Host-Network Traffic Management in Software-Defined Networks. J Netw Syst Manage 23, 374–399 (2015). https://doi.org/10.1007/s10922-014-9321-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10922-014-9321-9

Keywords

Search

Navigation