Abstract
Zero-day (or unknown) traffic brings about challenges for network security and management tasks, in terms of identifying the occurrence of those events in the network in an accurate and timely manner. In this paper, we propose a distributed mechanism to detect such unknown traffic in a timely manner. We compare our distributed scheme with a centralized system, where all the network flow data are used as a whole to perform the detection. We combined supervised and unsupervised learning mechanisms to discover and classify the unknown traffic efficiently, using clustering and Random Forest (RF) based schemes for this purpose. Further, we incorporated the correlation information in the traffic flows to improve the accuracy of detection, by means of using a Bag of Flows (BoFs) based method. Evaluation on real traces reveal that our distributed approach achieves a comparable detection performance to that of a centralized scheme. Further, the distributed scheme that incorporates unknown sample sharing in the framework shows improvement in the zero-day traffic detection performance. Moreover, the classifier used with the combination of BoF and RF shows improved detection accuracy, compared with not using BoFs.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Nguyen, T.T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutor. 10(4), 56–76 (2008)
Finamore, A., Mellia, M., Meo, M., Rossi, D.: KISS: stochastic packet inspection classifier for UDP traffic. IEEE/ACM Trans. Netw. (TON) 18(5), 1505–1515 (2010)
Juvonen, A., Sipola, T.: Adaptive framework for network traffic classification using dimensionality reduction and clustering. In: 2012 4th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), pp. 274–279. IEEE (2012)
Kim, H., Claffy, K.C., Fomenkov, M., Barman, D., Faloutsos, M., Lee, K.: Internet traffic classification demystified: myths, caveats, and the best practices. In: Proceedings of the 2008 ACM CoNEXT Conference, p. 11. ACM (2008)
Alazab, M., Venkatraman, S., Watters, P., Alazab, M.: Zero-day malware detection based on supervised learning algorithms of API call signatures. In: Proceedings of the Ninth Australasian Data Mining Conference, vol. 121, pp. 171–182. Australian Computer Society, Inc. (2011)
Este, A., Gringoli, F., Salgarelli, L.: Support vector machines for TCP traffic classification. Comput. Netw. 53(14), 2476–2490 (2009)
Finamore, A., Mellia, M., Meo, M.: Mining unclassified traffic using automatic clustering techniques. In: Domingo-Pascual, J., Shavitt, Y., Uhlig, S. (eds.) TMA 2011. LNCS, vol. 6613, pp. 150–163. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20305-3_13
Criminisi, A., Shotton, J., Konukoglu, E., et al.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends Comput. Graph. Vis. 7(2–3), 81–227 (2012)
Hastie, T., Tibshirani, R., Friedman, J.: Unsupervised learning. In: Hastie, T., Tibshirani, R., Friedman, J. (eds.) The Elements of Statistical Learning. Springer Series in Statistics, pp. 485–585. Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7_14
Zhang, J., Chen, X., Xiang, Y., Zhou, W., Wu, J.: Robust network traffic classification. IEEE/ACM Trans. Netw. (TON) 23(4), 1257–1270 (2015)
Miao, Y., Ruan, Z., Pan, L., Zhang, J., Xiang, Y., Wang, Y.: Comprehensive analysis of network traffic data. In: 2016 IEEE International Conference on Computer and Information Technology (CIT), pp. 423–430. IEEE (2016)
Han, Y., Chan, J., Alpcan, T., Leckie, C.: Using virtual machine allocation policies to defend against co-resident attacks in cloud computing. IEEE Trans. Dependable Secure Comput. 14(1), 95–108 (2017)
Rajasegarar, S., Leckie, C., Palaniswami, M.: Hyperspherical cluster based distributed anomaly detection in wireless sensor networks. J. Parallel Distrib. Comput. 74(1), 1833–1847 (2014)
Ling, Z., Luo, J., Wu, K., Yu, W., Fu, X.: Torward: discovery of malicious traffic over Tor. In: 2014 Proceedings IEEE INFOCOM, pp. 1402–1410. IEEE (2014)
Conti, M., Mancini, L.V., Spolaor, R., Verde, N.V.: Analyzing android encrypted network traffic to identify user actions. IEEE Trans. Inf. Forensics Secur. 11(1), 114–125 (2016)
Kharraz, A., Robertson, W., Balzarotti, D., Bilge, L., Kirda, E.: Cutting the gordian knot: a look under the hood of ransomware attacks. In: Almgren, M., Gulisano, V., Maggi, F. (eds.) DIMVA 2015. LNCS, vol. 9148, pp. 3–24. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20550-2_1
Zhang, J., Xiang, Y., Wang, Y., Zhou, W., Xiang, Y., Guan, Y.: Network traffic classification using correlation information. IEEE Trans. Parallel Distrib. Syst. 24(1), 104–117 (2013)
Zhang, J., Chen, C., Xiang, Y., Zhou, W., Vasilakos, A.V.: An effective network traffic classification method with unknown flow detection. IEEE Trans. Netw. Serv. Manag. 10(2), 133–147 (2013)
Erman, J., Mahanti, A., Arlitt, M.: QRP05-4: internet traffic identification using machine learning. In: IEEE GLOBECOM 2006, pp. 1–6, November 2006
Wang, Y., Xiang, Y., Yu, S.Z.: An automatic application signature construction system for unknown traffic. Concurr. Comput.: Pract. Exp. 22(13), 1927–1944 (2010)
Zhang, J., Chen, X., Xiang, Y., Zhou, W.: Zero-day traffic identification. In: Wang, G., Ray, I., Feng, D., Rajarajan, M. (eds.) CSS 2013. LNCS, vol. 8300, pp. 213–227. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03584-0_16
Acknowledgement
This work was supported by the National Natural Science Foundation of China under Grant 61401371.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Miao, Y., Pan, L., Rajasegarar, S., Zhang, J., Leckie, C., Xiang, Y. (2018). Distributed Detection of Zero-Day Network Traffic Flows. In: Boo, Y., Stirling, D., Chi, L., Liu, L., Ong, KL., Williams, G. (eds) Data Mining. AusDM 2017. Communications in Computer and Information Science, vol 845. Springer, Singapore. https://doi.org/10.1007/978-981-13-0292-3_11
Download citation
DOI: https://doi.org/10.1007/978-981-13-0292-3_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-0291-6
Online ISBN: 978-981-13-0292-3
eBook Packages: Computer ScienceComputer Science (R0)