Abstract
Recently many large scale computer systems are built in order to meet the high storage and processing demands of compute and data-intensive applications. MapReduce is one of the most popular programming models designed to support the development of such applications. It was initially created by Google for simplifying the development of large scale web search applications in data centers and has been proposed to form the basis of a ‘Data center computer’ This paper presents a realization of MapReduce for .NET-based data centers, including the programming model and the runtime system. The design and implementation of MapReduce.NET are described and its performance evaluation is presented.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
McNabb, A.W., Monson, C.K., Seppi, K.D.: Parallel PSO Using MapReduce. In: Proc. of the Congress on Evolutionary Computation (2007)
Apache. Hadoop, http://lucene.apache.org/hadoop/
Jin, C., Vecchiola, C., Buyya, R.: MRPGA: An Extension of MapReduce for Parallelizing Genetic Algorithms. In: Proc. of 4th International Conference on e-Science (2008)
Ranger, C., Raghuraman, R., Penmetsa, A., Bradski, G., Kozyrakis, C.: Evaluating MapReduce for Multi-core and Multiprocessor Systems. In: Proc. of the 13th Intl. Symposium on High-Performance Computer Architecture (2007)
Patterson, D.A.: Technical perspective: the data center is the computer. Communications of the ACMÂ 51(1), 105 (2008)
Gregor, D., Lumsdaine, A.: Design and Implementation of a High-Performance MPI for C\(\sharp\) and the Common Language Infrastructure. In: Proc. of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (2008)
Yang, H.C., Dasdan, A., Hsiao, R.L., Stott Parker, D.: Map-Reduce-Merge: simplified relational data processing on large clusters. In: Proc. of SIGMOD (2007)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: Proc. of the 6th Symposium on Operating System Design and Implementation (2004)
Varia, J.: Cloud Architectures. White Paper of Amazon (2008), jineshvaria.s3.amazonaws.com/public/cloudarchitectures-varia.pdf
Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks. In: Proc. of European Conference on Computer Systems, EuroSys (2007)
Kruijf, M., Sankaralingam, K.: MapReduce for the Cell B.E. Architecture. TR1625, Technical Report, The University of Wisconsin-Madison (2007)
Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud Computing and Emerging IT Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility. Future Generation Computer Systems 25(6), 599–616 (2009)
Bryant, R.E.: Data-Intensive Supercomputing: The Case for DISC. CMU-CS-07-128, Technical Report, Carnegie Mellon University (2007)
Chen, S., Schlosser, S.W.: Map-Reduce Meets Wider Varieties of Applications. IRP-TR-08-05, Technical Report, Intel Research Pittsburgh (2008)
Hey, T., Trefethen, A.: The data deluge: an e-Science perspective. In: Grid Computing: Making the Global Infrastructure a Reality, pp. 809–824 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jin, C., Buyya, R. (2009). MapReduce Programming Model for .NET-Based Cloud Computing. In: Sips, H., Epema, D., Lin, HX. (eds) Euro-Par 2009 Parallel Processing. Euro-Par 2009. Lecture Notes in Computer Science, vol 5704. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03869-3_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-03869-3_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03868-6
Online ISBN: 978-3-642-03869-3
eBook Packages: Computer ScienceComputer Science (R0)