skip to main content
research-article

InFeMo: Flexible Big Data Management Through a Federated Cloud System

Authors Info & Claims
Published:22 October 2021Publication History
Skip Abstract Section

Abstract

This paper introduces and describes a novel architecture scenario based on Cloud Computing and counts on the innovative model of Federated Learning. The proposed model is named Integrated Federated Model, with the acronym InFeMo. InFeMo incorporates all the existing Cloud models with a federated learning scenario, as well as other related technologies that may have integrated use with each other, offering a novel integrated scenario. In addition to this, the proposed model is motivated to deliver a more energy efficient system architecture and environment for the users, which aims to the scope of data management. Also, by applying the InFeMo the user would have less waiting time in every procedure queue. The proposed system was built on the resources made available by Cloud Service Providers (CSPs) and by using the PaaS (Platform as a Service) model, in order to be able to handle user requests better and faster. This research tries to fill a scientific gap in the field of federated Cloud systems. Thus, taking advantage of the existing scenarios of FedAvg and CO-OP, we were keen to end up with a new federated scenario that merges these two algorithms, and aiming for a more efficient model that is able to select, depending on the occasion, if it “trains” the model locally in client or globally in server.

REFERENCES

  1. [1] Stergiou C. and Psannis K. E.. 2017. Algorithms for Big Data in advanced communication systems and Cloud computing. In Proceedings of 19th IEEE Conference on Business Informatics 2017 (CBI2017), Doctoral Consortium 24–26 July 2017, Thessaloniki, Greece. DOI: DOI: 10.1109/CBI.2017.28Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Marr B.. 2014. Big Data: The 5 Vs everyone must know. LinkedIn article 6 March 2014. Retrieved December 17, 2018 from https://www.linkedin.com/pulse/20140306073407-64875646-big-data-the-5-vs-everyone-must-know.Google ScholarGoogle Scholar
  3. [3] Lv Z. and Singh A. K.. 2020. Big Data analysis of Internet of Things system. ACM Transactions on Internet Technology 0, ja, Accepted on March 2020. DOI: DOI: 10.1145/3389250Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Stergiou C. and Psannis K. E.. 2016. Recent advances delivered by mobile cloud computing and Internet of Things for Big Data applications: A survey. Wiley Online Library, International Journal of Network Management 27, 3 (May 2016), 112.Google ScholarGoogle Scholar
  5. [5] Rathore M. M., Paul A., Ahmad A., Anisetti M., and Jeon G.. 2017. Hadoop-based intelligent care system (HICS): Analytical approach for big data in IoT. ACM Transactions on Internet Technology 18, 1, No. 8, 24 pages, November 2017. DOI: DOI: 10.1145/3108936 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Yu H., Yang J., and Fung C.. 2020. Fine-grained Cloud resource provisioning for virtual network function. IEEE Transactions on Network and Service Management, In Press 2020.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Stergiou C. and Psannis K. E.. Efficient and secure Big Data delivery in Cloud computing. Springer, Multimedia Tools and Applications 76, 21 (November 2017), 2280322822. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Stergiou C., Psannis K. E., Kim B.-G., and Gupta B.. 2018. Secure integration of IoT and Cloud computing. 2018. Elsevier, Future Generation Computer Systems 78, part 3 (January 2018), 964975. DOI: DOI: 10.1016/j.future.2016.11.031Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Hilbert M. and López P.. 2011. The world's technological capacity to store, communicate, and compute information. Science 332, 6025 (April 2011), 6065. DOI: DOI: 10.1126/science.1200970Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Fu Z., Ren K., Shu J., Sun X., and Huang F.. 2016. Enabling personalized search over encrypted outsourced data with efficiency improvement. IEEE Transactions on Parallel and Distributed Systems 27, 9, September 2016. DOI: DOI: 10.1109/TPDS.2015.2506573 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Agrawal D., Das S., and El Abbadi A.. 2011. Big Data and cloud computing: Current state and future opportunities. In Proceedings of 14th International Conference on Extending Database Technology, EDBT 2011, 2124 March 2011, Uppsala, Sweden, pp. 530533. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Pahl C., Jamshidi P., and Zimmermann O.. 2018. Architectural principles for Cloud software. ACM Transactions on Internet Technology 18, 2, No. 17, 23 pages, February 2018. DOI: DOI: 10.1145/3104028 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Ferry N., Chauvel F., Song H., Rossini A., Lushpenko M., and Solberg A.. 2018. CloudMF: Model-driven management of multi-cloud applications. ACM Transactions on Internet Technology 18, 2, No. 16 (January 2018), 23 pages. DOI: DOI: 10.1145/3125621 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Yao X., Huang C., and Sun L.. 2018. Two-stream federated learning: Reduce the communication costs. In Proceedings of 2018 IEEE Visual Communications and Image Processing (VCIP) 9-12 December 2018, Taichung, Taiwan, Taiwan. DOI: DOI: 10.1109/VCIP.2018.8698609Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Nilsson A., Smith S., Ulm G., Gustavsson E., and Jirstrand M.. 2018. A performance evaluation of federated learning algorithms. In Proceedings of DIDL'18: Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning December 2018, pp. 18, Middleware'18: 19th International Middleware Conference Rennes France. DOI: DOI: 10.1145/3286490.3286559 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] McMahan H. B., Moore E., Ramage D., Hampson S., and y Arcas B. A.. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017, JMLR: W&CP volume 54, 2022 April 2017, Fort Lauderdale, Florida, USA. arXiv:1602. 05629Google ScholarGoogle Scholar
  17. [17] Shokri R. and Shmatikov V.. 2015. Privacy-preserving deep learning. In Proceedings of 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton) 30 September –2 October 2015, Allerton Park and Conference Center, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Thakur S. and Breslin J. G.. 2019. A robust reputation management mechanism in the federated Cloud. IEEE Transactions on Cloud Computing 7, 3 (July-September 2019), 625637. DOI: DOI: 10.1109/TCC.2017.2689020Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Cai Q., Zhang H., Guo W., Chen G., Ooi B. Chin, Tan K.-L., and Wong W.-F.. 2019. MemepiC: Towards a unified in-memory Big Data management system. IEEE Transactions on Big Data 5, 1 (March 2019), 417. DOI: DOI: 10.1109/TBDATA.2017.2789286Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Pasquier T. F. J.-M., Singh J., Eyers D., and Bacon J.. 2017. CamFlow: Managed data-sharing for Cloud services. IEEE Transactions on Cloud Computing 5, 3 (July-September 2017), 472484. DOI: DOI: 10.1109/TCC.2015.2489211Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Zhu L., Wu Y., Gai K., and Choo K.-K. R.. 2019. Controllable and trustworthy blockchain-based Cloud data management. Elsevier, Future Generation Computer Systems 91 (February 2019), 527535. DOI: DOI: 10.1016/j.future.2018.09.019Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Yan Z., Zhang L., Ding W., and Zheng Q.. 2019. Heterogeneous data storage management with deduplication in Cloud computing. IEEE Transactions on Big Data 5, 3 (September 2019), 393407. DOI: DOI: 10.1109/TBDATA.2017.2701352Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Premarathne U. S., Khalil I., Tari Z., and Zomaya A.. 2017. Cloud-based utility service framework for trust negotiations using federated identity management. IEEE Transactions on Cloud Computing 5, 2 (April-June 2017), 290302. DOI: DOI: 10.1109/TCC.2015.2404816Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Mansha S. and Kamiran F.. 2015. Multi-query optimization in federated databases using evolutionary algorithm. In Proceedings of the 2015 IEEE 14th International Conference on Machine Learning and Applications 9-11 December 2015, Miami, FL, USA. DOI: DOI: 10.1109/ICMLA.2015.125Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Wang S., Tuor T., Salonidis T., Leung K. K., Makaya C., He T., and Chan K.. 2019. Adaptive federated learning in resource constrained edge computing systems. IEEE Journal on Selected Areas in Communications, ver. 99, pp. 11, March 2019. DOI: DOI: 10.1109/JSAC.2019.2904348Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Brendan McMahan H., Moore Eider, Ramage Daniel, Hampson Seth et al. 2016. Communication-efficient learning of deep networks from decentralized data. arXiv: 1602.05629Google ScholarGoogle Scholar
  27. [27] Wang Yushi. 2017. CO-OP: Cooperative machine learning from mobile devices. Master's thesis. Dept. Elect. And Comput. Eng., Univ. Alberta, Edmonton, Canada.Google ScholarGoogle Scholar
  28. [28] Young B., Bhatnagar R., Tatavarty G., and Bian H.. 2007. Covariance matrix computations with federated databases. In Proceedings of ICMLA'07: Proceedings of the Sixth International Conference on Machine Learning and Applications 13–15 December 2007, pp. 172177, Cincinnati, OH, USA. DOI: DOI: 10.1109/ICMLA.2007.36 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Konecny J., McMahan H. B., and Ramage D.. 2015. Federated optimization: Distributed optimization beyond the datacenter. ArXiv, pp. 138, November, 2015. arXiv:1511.03575 and Retrieved March 2020 from https://arxiv.org/abs/1511.03575.Google ScholarGoogle Scholar
  30. [30] Pei J., Hong P., Xue K., and Li D.. 2019. Efficiently embedding service function chains with dynamic virtual network function placement in geo-distributed cloud system. IEEE Transactions on Parallel and Distributed Systems 30, 10 (October 2019), 21792192. DOI: DOI: 10.1109/TPDS.2018.2880992Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Chen K.-Y., Xu Y., Xi K., and Chao H. J.. 2013. Intelligent virtual machine placement for cost efficiency in geo-distributed Cloud systems. In Proceedings of 2013 IEEE International Conference on Communications (ICC), 913 June 2013, Budapest, Hungary. DOI: DOI: 10.1109/ICC.2013.6655092Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Konecny Jakub, McMahan H. Brendan, Ramage Daniel, and Richtarik Peter. 2016. Federated optimization: Distributed machine learning for on-device intelligence. arXiv: 1610.02527Google ScholarGoogle Scholar
  33. [33] European Commission. 2018. What data can we process and under which conditions? Retrieved March 14, 2020 from https://ec.europa.eu/info/law/law-topic/data-protection/reform/rules-business-and-organisations/principles-gdpr/what-data-can-we-process-and-under-which-conditions_en.Google ScholarGoogle Scholar

Index Terms

  1. InFeMo: Flexible Big Data Management Through a Federated Cloud System

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Internet Technology
        ACM Transactions on Internet Technology  Volume 22, Issue 2
        May 2022
        582 pages
        ISSN:1533-5399
        EISSN:1557-6051
        DOI:10.1145/3490674
        • Editor:
        • Ling Liu
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 22 October 2021
        • Accepted: 1 October 2021
        • Revised: 1 September 2020
        • Received: 1 July 2020
        Published in toit Volume 22, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format