Abstract
The objective of this paper is to present a SMACK based platform for microgrids data storage and management. The platform is being used in a real microgrid, with an infrastructure that monitors and controls 3 buildings within the GECAD - ISEP/IPP campus, while, at the same time, receives and manages data sources coming from different types of buildings from associated partners, to whom intelligent services are being provided. Microgrid data comes in different formats, different rates and with an increasing volume, as the microgrid itself covers more customers and areas. Based on the atual available computational resources, a Big Data platform based on the SMACK stack was implemented and is presented. The Cassandra component of the stack has evolved. AC version 2 is still supported until the version 4 release, and is often still used in production environments. However, a new stable version, version 3, introduces major optimizations in the storage that bring disk space savings. The main focus of this work is on the Data Storage and the formalization of the data mapping in Cassandra version 3, which is contextualized by means of a short example with data coming from the monitoring infrastructure of the microgrid.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
1. Skopik, F., Ma, Z.: Attack Vectors to Metering Data in Smart Grids under Security Constraints. In: 2012 IEEE 36th Annual Computer Software and Applications Conference Workshops. pp. 134–139. IEEE (2012).
2. Chandarana, P., Vijayalakshmi, M.: Big Data analytics frameworks. In: 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA). pp. 430–434. IEEE (2014).
3. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop Distributed File System. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). pp. 1–10. IEEE (2010).
4. Zaharia, M., Franklin, M.J., Ghodsi, A., Gonzalez, J., Shenker, S., Stoica, I., Xin, R.S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S.: Apache Spark: A Unified Engine for Big Data Processing. Commun. ACM. 59, 56–65 (2016).
5. Felix Covrig, C., Ardelean, M., Vasiljevska, J., Mengolini, A., Fulli, G., Amoiralis, E., Sánchez Jiménez, M., Filiou, C., European Commission. Joint Research Centre. Institute for Energy and Transport.: Smart grid projects outlook 2014. Publications Office (2014).
6. Vinagre, E., Gomes, L., Vale, Z.: Electrical energy consumption forecast using external facility data. Proc. - 2015 IEEE Symp. Ser. Comput. Intell. SSCI 2015. 659–664 (2016).
7. Gomes, L., Lefrancois, M., Faria, P., Vale, Z.: Publishing real-time microgrid consumption data on the web of Linked Data. In: 2016 Clemson University Power Systems Conference (PSC). pp. 1–8. IEEE (2016).
8. Estrada, R., Ruiz, I.: Big Data SMACK. Apress, Berkeley, CA (2016).
9. Mayilvaganan, M., Sabitha, M.: A cloud-based architecture for Big-Data analytics in smart grid: A proposal. In: 2013 IEEE International Conference on Computational Intelligence and Computing Research. pp. 1–4. IEEE (2013).
10. Zhou, D., Guo, J., Zhang, Y., Chai, J., Liu, H., Liu, Y., Huang, C., Gui, X., Liu, Y.: Distributed Data Analytics Platform for Wide-Area Synchrophasor Measurement Systems. IEEE Trans. Smart Grid. 1–9 (2016).
11. Chebotko, A., Kashlev, A., Lu, S.: A Big Data Modeling Methodology for Apache Cassandra. In: 2015 IEEE International Congress on Big Data. pp. 238–245. IEEE (2015).
12. Datastax: Estimating partition size, https://docs.datastax.com/en/landing_page/doc/landing_page/planning/planningPartitionSize.html. Accessed 23 Nov 2016
13. Datastax Academy: Physical: Partition Size, https://academy.datastax.com/courses/ds220-data-modeling/physical-partition-size. Accessed 20 Nov 2016
14. Lebresne, S.: Putting some structure in the storage engine, http://www.datastax.com/2015/12/storage-engine-30. Accessed 4 Dec 2016
15. Morton, A.: Introduction To The Apache Cassandra 3.x Storage Engine, http://thelastpickle.com/blog/2016/03/04/introductiont-to-the-apache-cassandra-3-storage-engine.html. Accessed 4 Dec 2016
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Pinheiro, G., Vinagre, E., Praça, I., Vale, Z., Ramos, C. (2018). Smart Grids Data Management: A Case for Cassandra. In: Omatu, S., Rodríguez, S., Villarrubia, G., Faria, P., Sitek, P., Prieto, J. (eds) Distributed Computing and Artificial Intelligence, 14th International Conference. DCAI 2017. Advances in Intelligent Systems and Computing, vol 620. Springer, Cham. https://doi.org/10.1007/978-3-319-62410-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-62410-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-62409-9
Online ISBN: 978-3-319-62410-5
eBook Packages: EngineeringEngineering (R0)