Jure Leskovec
Jon Kleinberg
Christos Faloutsos
Abstract
How do real graphs evolve over time? What are normal growth patterns in social, technological, and information networks? Many studies have discovered patterns in static graphs, identifying properties in a single snapshot of a large network or in a very small number of snapshots; these include heavy tails for in- and out-degree distributions, communities, small-world phenomena, and others. However, given the lack of information about network evolution over long periods, it has been hard to convert these findings into statements about trends over time.
Here we study a wide range of real graphs, and we observe some surprising phenomena. First, most of these graphs densify over time with the number of edges growing superlinearly in the number of nodes. Second, the average distance between nodes often shrinks over time in contrast to the conventional wisdom that such distance parameters should increase slowly as a function of the number of nodes (like O(log n) or O(log(log n)).
Existing graph generation models do not exhibit these types of behavior even at a qualitative level. We provide a new graph generator, based on a forest fire spreading process that has a simple, intuitive justification, requires very few parameters (like the flammability of nodes), and produces graphs exhibiting the full range of properties observed both in prior work and in the present study.
We also notice that the forest fire model exhibits a sharp transition between sparse graphs and graphs that are densifying. Graphs with decreasing distance between the nodes are generated around this transition point.
Last, we analyze the connection between the temporal evolution of the degree distribution and densification of a graph. We find that the two are fundamentally related. We also observe that real networks exhibit this type of relation between densification and the degree distribution.
- Abello, J. 2004. Hierarchical graph maps. Comput. Graph. 28, 3, 345--359.]]Google Scholar
Cross Ref
- Abello, J., Buchsbaum, A. L., and Westbrook, J. 1998. A functional approach to external graph algorithms. In Proceedings of the 6th Annual European Symposium on Algorithms. Springer-Verlag, 332--343.]] Google ScholarDigital Library
- Abello, J., Pardalos, P. M., and Resende, M. G. C. 2002. Handbook of Massive Data Sets. Kluwer Academic Publishing.]] Google ScholarDigital Library
- Adamic, L. A. 2000. Zipf, power-law, pareto---a ranking tutorial. http://www.hpl.hp.com/research/idl/papers/ranking.]]Google Scholar
- Albert, R. and Barabasi, A.-L. 1999. Emergence of scaling in random networks. Science. 509--512.]]Google Scholar
- Albert, R., Jeong, H., and Barabasi, A.-L. 1999. Diameter of the World-Wide Web. Nature 401, 130--131.]]Google ScholarCross Ref
- Alderson, D., Doyle, J. C., Li, L., and Willinger, W. 2005. Towards a theory of scale-free graphs: Definition, properties, and implications. Internet Math. 2, 4.]]Google Scholar
- Bi, Z., Faloutsos, C., and Korn, F. 2001. The dgx distribution for mining massive, skewed data. In Proceedings of Knowledge Discovery and Data Mining (KDD). 17--26.]] Google ScholarDigital Library
- Bollobas, B. and Riordan, O. 2004. The diameter of a scale-free random graph. Combinatorica 24, 1, 5--34.]] Google ScholarDigital Library
- Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., and Wiener, J. 2000a. Graph structure in the Web. In Proceedings of the 9th International World Wide Web Conference on Computer Networks: The International Journal of Computer and Telecommunications Netowrking. North-Holland Publishing Co., Amsterdam, The Netherlands, 309--320.]] Google ScholarDigital Library
- Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., and Wiener, J. 2000b. Graph structure in the web: experiments and models. In Proceedings of World Wide Web Conference.]] Google ScholarDigital Library
- Chakrabarti, D. and Faloutsos, C. 2006. Graph mining: Laws, generators, and algorithms. ACM Comput. Sur. 38, 1.]] Google ScholarDigital Library
- Chakrabarti, D., Zhan, Y., and Faloutsos, C. 2004. R-mat: A recursive model for graph mining. In Proceedings of the SIAM Conference on Data Mining (SDM).]]Google Scholar
- Chung, F. and Lu, L. 2002. The average distances in random graphs with given expected degrees. Proceedings of the National Academy of Sciences 99, 25, 15879--15882.]]Google ScholarCross Ref
- Cooper, C. and Frieze, A. 2003. A general model of web graphs. Random Struct. Algo. 22, 3, 311--335.]] Google ScholarDigital Library
- Dorogovtsev, S. and Mendes, J. 2001a. Effect of the accelerated growth of communications networks on their structure. Phys. Rev. E 63, 025101.]]Google ScholarCross Ref
- Dorogovtsev, S. and Mendes, J. 2001b. Language as an evolving word web. Proceedings of the Royal Society of London B 268, 2603.]]Google ScholarCross Ref
- Dorogovtsev, S. and Mendes, J. 2002. Accelerated growth of networks. In Handbook of Graphs and Networks: From the Genome to the Internet, S. Bornholdt and H.G. Schuster. Eds. Wiley-VCH, Berlin, Germany.]]Google Scholar
- Dorogovtsev, S. and Mendes, J. 2003. Evolution of Networks: From Biological Nets to the Internet and WWW. Oxford University Press, Oxford, UK.]] Google ScholarDigital Library
- Faloutsos, M., Faloutsos, P., and Faloutsos, C. 1999. On power-law relationships of the Internet topology. In SIGCOMM. 251--262.]] Google ScholarDigital Library
- Gehrke, J., Ginsparg, P., and Kleinberg, J. M. 2003. Overview of the 2003 kdd cup. SIGKDD Explora. 5, 2, 149--151.]] Google ScholarDigital Library
- Hall, B. H., Jaffe, A. B., and Trajtenberg, M. 2001. The nber patent citation data file: Lessons, insights and methodological tools. NBER Working Papers 8498, National Bureau of Economic Research, Inc. (Oct.)]]Google Scholar
- Huberman, B. A. and Adamic, L. A. 1999. Growth dynamics of the world-wide web. Nature 399, 131.]]Google ScholarCross Ref
- Katz, J. S. 1999. The self-similar science system. Resear. Policy 28, 501--517.]]Google ScholarCross Ref
- Katz, J. S. 2005. Scale independent bibliometric indicators. Measure.: Interdisciplin. Resea. Perspect. 3, 24--28.]]Google ScholarCross Ref
- Kleinberg, J. M. 2002. Small-world phenomena and the dynamics of information. In Advances in Neural Information Processing Systems 14.]]Google Scholar
- Kleinberg, J. M., Kumar, R., Raghavan, P., Rajagopalan, S., and Tomkins, A. 1999. The Web as a graph: Measurements, models, and methods. In Proceedings of the International Conference on Combinatorics and Computing. 1--17.]]Google Scholar
- Kossinets, G. and Watts, D. J. 2006. Empirical analysis of an evolving social network. Science 311, 88--90.]]Google ScholarCross Ref
- Krapivsky, P. L. and Redner, S. 2001. Organization of growing random networks. Phys. Rev. E 63, 066123.]]Google ScholarCross Ref
- Krapivsky, P. L. and Redner, S. 2005. Network growth by copying. Phys. Rev. E 71, 036118.]]Google ScholarCross Ref
- Kumar, R., Raghavan, P., Rajagopalan, S., Sivakumar, D., Tomkins, A., and Upfal, E. 2000. Stochastic models for the web graph. In Proceedings of the 41st IEEE Symposium on Foundations of Computer Science.]] Google ScholarDigital Library
- Kumar, R., Raghavan, P., Rajagopalan, S., and Tomkins, A. 1999. Trawling the Web for emerging cyber-communities. In Proceedings of the 8th International World Wide Web Conference.]] Google ScholarDigital Library
- Leskovec, J., Adamic, L., and Huberman, B. 2006. The dynamics of viral marketing. ACM Conference on Electronic Commerce.]] Google ScholarDigital Library
- Leskovec, J., Chakrabarti, D., Kleinberg, J. M., and Faloutsos, C. 2005. Realistic, mathematically tractable graph generation and evolution, using kronecker multiplication. In Proceedings of the European International Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD'05). 133--145.]]Google Scholar
- Leskovec, J. and Faloutsos, C. 2006. Sampling from large graphs. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'06). ACM Press, New York, NY, USA, 631--636.]] Google ScholarDigital Library
- Leskovec, J., Kleinberg, J., and Faloutsos, C. 2005. Graphs over time: densification laws, shrinking diameters and possible explanations. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD'05). Chicago, IL.]] Google ScholarDigital Library
- Menczer, F. 2002. Growing and navigating the small world web by local content. Proceedings of the National Academy of Sciences 99, 22, 14014--14019.]]Google ScholarCross Ref
- Milgram, S. 1967. The small-world problem. Psycholo. Today 2, 60--67.]]Google Scholar
- Mitzenmacher, M. 2004. A brief history of generative models for power law and lognormal distributions. Internet Math. 1, 2, 226--251.]]Google ScholarCross Ref
- Newman, M. E. J. 2003. The structure and function of complex networks. SIAM Review 45, 167--256.]]Google ScholarDigital Library
- Newman, M. E. J. 2005. Power laws, pareto distributions and zipf's law. Contemp. Phys. 46, 323--351.]]Google ScholarCross Ref
- Ntoulas, A., Cho, J., and Olston, C. 2004. What's new on the web? the evolution of the web from a search engine perspective. In World Wide Web Conference. New York, 1--12.]] Google ScholarDigital Library
- Oregon. 1997. University of Oregon route views project. online data and reports. http://www.routeviews.org.]]Google Scholar
- Palmer, C. R., Gibbons, P. B., and Faloutsos, C. 2002. Anf: A fast and scalable tool for data mining in massive graphs. In SIGKDD. Edmonton, AB, Canada.]] Google ScholarDigital Library
- Redner, S. 2004. Citation statistics from more than a century of physical review. Tech. rep. physics/0407137, arXiv.]]Google Scholar
- Schroeder, M. 1991. Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise. W.H. Freeman and Company, New York, NY.]]Google Scholar
- Tauro, S. L., Palmer, C., Siganos, G., and Faloutsos, M. 2001. A simple conceptual model for the internet topology. In Global Internet. San Antonio, TX.]]Google Scholar
- Vazquez, A. 2001. Disordered networks generated by recursive searches. Europhy. Lett. 54, 4, 430--435.]]Google Scholar
- Vazquez, A. 2003. Growing networks with local rules: Preferential attachment, clustering hierarchy and degree correlations. Physical Review E 67, 056104.]]Google ScholarCross Ref
- Watts, D. J., Dodds, P. S., and Newman, M. E. J. 1998. Collective dynamics of ‘small-world’ networks. Nature 393, 440--442.]]Google ScholarCross Ref
- Watts, D. J., Dodds, P. S., and Newman, M. E. J. 2002. Identity and search in social networks. Science 296, 1302--1305.]]Google ScholarCross Ref
Index Terms
- Graph evolution: Densification and shrinking diameters
Recommendations
Graphs over time: densification laws, shrinking diameters and possible explanations
KDD '05: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data miningHow do real graphs evolve over time? What are "normal" growth patterns in social, technological, and information networks? Many studies have discovered patterns in static graphs, identifying properties in a single snapshot of a large network, or in a ...
Agwan: a generative model for labelled, weighted graphs
NFMCP'13: Proceedings of the 2nd International Conference on New Frontiers in Mining Complex PatternsReal-world graphs or networks tend to exhibit a well-known set of properties, such as heavy-tailed degree distributions, clustering and community formation. Much effort has been directed into creating realistic and tractable models for unlabelled graphs,...
Same Stats, Different Graphs: (Graph Statistics and Why We Need Graph Drawings)
Graph Drawing and Network VisualizationAbstractData analysts commonly utilize statistics to summarize large datasets. While it is often sufficient to explore only the summary statistics of a dataset (e.g., min/mean/max), Anscombe’s Quartet demonstrates how such statistics can be misleading. We ...
Comments