Hostname: page-component-848d4c4894-wg55d Total loading time: 0 Render date: 2024-05-16T22:42:48.078Z Has data issue: false hasContentIssue false
  • English
  • Français

Testing Social Science Network Theories with Online Network Data: An Evaluation of External Validity

Published online by Cambridge University Press:  13 June 2017

JAMES BISBEE  and
JENNIFER M. LARSON
JAMES BISBEE*
Affiliation:
New York University
JENNIFER M. LARSON*
Affiliation:
New York University
*
James Bisbee is Ph.D. Candidate, NYU Department of Politics, 19 W. 4th St., New York, NY 10012. (james.bisbee@nyu.edu).
Jennifer M. Larson is Assistant Professor, NYU Department of Politics, 19 W. 4th St., New York, NY 10012. Corresponding Author (jenn.larson@nyu.edu).
Get access Rights & Permissions [Opens in a new window]

Abstract

To answer questions about the origins and outcomes of collective action, political scientists increasingly turn to datasets with social network information culled from online sources. However, a fundamental question of external validity remains untested: are the relationships measured between a person and her online peers informative of the kind of offline, “real-world” relationships to which network theories typically speak? This article offers the first direct comparison of the nature and consequences of online and offline social ties, using data collected via a novel network elicitation technique in an experimental setting. We document strong, robust similarity between online and offline relationships. This parity is not driven by shared identity of online and offline ties, but a shared nature of relationships in both domains. Our results affirm that online social tie data offer great promise for testing long-standing theories in the social sciences about the role of social networks.

Type
Research Article
Information
American Political Science Review , Volume 111 , Issue 3 , August 2017 , pp. 502 - 521
Copyright
Copyright © American Political Science Association 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors thank Neal Beck, Eric Dickson, Sean Kates, and Xiang Zhou, as well as the participants of the New York University Empirical Research for Advanced Students seminar and the participants of the 2017 Harvard Experimental Political Science Graduate Student Conference, for their valuable feedback on the project.

References

REFERENCES

Acemoglu, Daron, Ozdaglar, Asuman, and ParandehGheibi, Ali. 2010. “Spread of (Mis) Information in Social Networks.” Games and Economic Behavior 70 (2): 194–227.Google Scholar
Ahn, Yong-Yeol, Han, Seungyeop, Kwak, Haewoon, Moon, Sue, and Jeong, Hawoong. 2007. “Analysis of Topological Characteristics of Huge Online Social Networking Services.” In Proceedings of the 16th International Conference on World Wide Web. New York, NY: ACM Press, 835–44.Google Scholar
Amir, Ofra, Rand, David G., and Gal, Ya’akov Kobi. 2012. “Economic Games on the Internet: The Effect of $1 Stakes.” PLoS One 7 (2): e31461.Google Scholar
Aral, Sinan, and Walker, Dylan. 2011. “Creating Social Contagion Through Viral Product Design: A Randomized Trial of Peer Influence in Networks.” Management Science 57 (9): 1623–39.Google Scholar
Bartneck, Christoph, Duenser, Andreas, Moltchanova, Elena, and Zawieska, Karolina. 2015. “Comparing the Similarity of Responses Received from Studies in Amazon’s Mechanical Turk to Studies Conducted Online and with Direct Recruitment.” PloS One 10 (4): e0121595.Google Scholar
Berinsky, Adam J., Huber, Gregory A., and Lenz, Gabriel S.. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk.” Political Analysis 20 (3): 351–68.Google Scholar
Bond, Robert M., Fariss, Christopher J., Jones, Jason J., Kramer, Adam D. I., Marlow, Cameron, Settle, Jaime E., and Fowler, James H.. 2012. “A 61-Million-Person Experiment in Social Influence and Political Mobilization.” Nature 489 (7415): 295–98.Google Scholar
Bramoullé, Yann, and Kranton, Rachel. 2007. “Public Goods in Networks.” Journal of Economic Theory 135 (1): 478–94.Google Scholar
Calvo-Armengol, Antoni, and Jackson, Matthew O.. 2004. “The Effects of Social Networks on Employment and Inequality.” American Economic Review 94 (3): 426–54.Google Scholar
Centola, Damon. 2010. “The Spread of Behavior in an Online Social Network Experiment.” Science 329 (5996): 1194–97.Google Scholar
Centola, Damon, and Macy, Michael. 2007. “Complex Contagions and the Weakness of Long Ties.” American Journal of Sociology 113 (3): 702–34.Google Scholar
Chandler, Jesse, and Shapiro, Danielle. 2016. “Conducting Clinical Science Research on Amazon Mechanical Turk.” Annual Review of Clinical Psychology 12 (1): 3–5.Google Scholar
Christakis, Nicholas A., and Fowler, James H.. 2013. “Social Contagion Theory: Examining Dynamic Social Networks and Human Behavior.” Statistics in Medicine 32 (4): 556–77.Google Scholar
Chwe, Michael Suk-Young. 2000. “Communication and Coordination in Social Networks.” Review of Economic Studies 67 (1): 1– 16.Google Scholar
Clifford, Scott, Jewell, Ryan M., and Waggoner, Philip D.. 2015. “Are Samples Drawn from Mechanical Turk Valid for Research on Political Methodology?” Research and Politics 2 (4): 205316801562272.Google Scholar
Conley, Timothy G., and Udry, Christopher R.. 2010. “Learning About a New Technology: Pineapple in Ghana.” American Economic Review 100 (1): 35–69.Google Scholar
Crump, Matthew J. C., McDonnell, John V., and Gureckis, Todd M.. 2013. “Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research.” PloS One 8 (3): e57410.Google Scholar
Ellison, Nicole B., Steinfield, Charles, and Lampe, Cliff. 2007. “The Benefits of Facebook ‘Friends:’ Social Capital and College Students’ Use of Online Social Network Sites.” Journal of Computer-Mediated Communication 12 (4): 1143–68.CrossRefGoogle Scholar
Ferrali, R. 2016. “Partners in Crime? A Theory of Corruption as a Criminal Network.” Working Paper. http://scholar.princeton.edu/sites/default/files/ferrali/files/ferrali_corruption.pdf.Google Scholar
Gilbert, Eric, and Karahalios, Karrie. 2009. “Predicting Tie Strength with Social Media.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM Press, 211–20.Google Scholar
González-Bailón, Sandra, and Wang, Ning. 2016. “Networked Discontent: The Anatomy of Protest Campaigns in Social Media.” Social Networks 44: 95–104.Google Scholar
Goodman, Joseph K., Cryder, Cynthia E., and Cheema, Amar. 2013. “Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples.” Journal of Behavioral Decision Making 26 (3): 213–24.Google Scholar
Grabowicz, Przemyslaw A., Ramasco, José J., Moro, Esteban, Pujol, Josep M., and Eguiluz, Victor M.. 2012. “Social Features of Online Networks: The Strength of Intermediary Ties in Online Social Media.” PloS One 7 (1): e29358.Google Scholar
Granovetter, Mark S. 1973. “The Strength of Weak Ties.” American Journal of Sociology 78 (6): 1360–80.Google Scholar
Horton, John J., Rand, David G., and Zeckhauser, Richard J.. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399–425.Google Scholar
Huff, Connor, and Tingley, Dustin. 2015. “‘Who Are These People?’ Evaluating the Demographic Characteristics and Political Preferences of MTurk Survey Respondents.” Research and Politics 2 (1): 1–12.Google Scholar
Jackson, Matthew O., Rodriguez-Barraquer, Tomas, and Tan, Xu. 2012. “Social Capital and Social Quilts: Network Patterns of Favor Exchange.” American Economic Review 102 (5): 1857–97.Google Scholar
Joinson, Adam. 1999. “Social Desirability, Anonymity, and Internet-Based Questionnaires.” Behavior Research Methods, Instruments, and Computers 31 (3): 433–38.Google Scholar
Jones, Jason J., Settle, Jaime E., Bond, Robert M., Fariss, Christopher J., Marlow, Cameron, and Fowler, James H.. 2013. “Inferring Tie Strength from Online Directed Behavior.” PloS One 8 (1): e52168.Google Scholar
Larson, Jennifer M. 2016. “Interethnic Conflict and the Potential Dangers of Cross-Group Ties.” Journal of Peace Research 53 (3): 459–71.Google Scholar
Larson, Jennifer M. 2017. “Networks and Interethnic Cooperation.” Journal of Politics 79 (2): 546–59.Google Scholar
Larson, Jennifer M., and Lewis, Janet I.. 2017. “Measuring Networks in the Field.” Working Paper. http://goo.gl/cRWP3j.Google Scholar
Lazer, David, Pentland, Alex Sandy, Adamic, Lada, Aral, Sinan, Barabasi, Albert Laszlo, Brewer, Devon, Christakis, Nicholas, Contractor, Noshir, Fowler, James, Gutmann, Myron, et al. 2009. “Life in the Network: The Coming Age of Computational Social Science.” Science 323 (5915): 721.Google Scholar
Levay, Kevin E., Freese, Jeremy, and Druckman, James N.. 2016. “The Demographic and Political Composition of Mechanical Turk Samples.” SAGE Open 6 (1): 1–16.Google Scholar
Marwell, Gerald, Oliver, Pamela E., and Prahl, Ralph. 1988. “Social Networks and Collective Action: A Theory of the Critical Mass. III.” American Journal of Sociology 94 (3): 502–34.Google Scholar
McAdam, Doug, and Paulsen, Ronnelle. 1993. “Specifying the Relationship Between Social Ties and Activism.” American Journal of Sociology 99 (3): 640–67.Google Scholar
McClurg, Scott D. 2006. “The Electoral Relevance of Political Talk: Examining Disagreement and Expertise Effects in Social Networks on Political Participation.” American Journal of Political Science 50 (3): 737–54.Google Scholar
Milgram, Stanley, and Gudehus, Christian. 1978. Obedience to Authority. New York, NY: Ziff-Davis Publishing Company.Google Scholar
Opp, Karl-Dieter, and Gern, Christiane. 1993. “Dissident Groups, Personal Networks, and Spontaneous Cooperation: The East German Revolution of 1989.” American Sociological Review 58 (5): 659–80.Google Scholar
Paolacci, Gabriele, Chandler, Jesse, and Ipeirotis, Panagiotis G.. 2010. “Running Experiments on Amazon Mechanical Turk.” Judgment and Decision Making 5 (5): 411–19.Google Scholar
Parkinson, Sarah Elizabeth. 2013. “Organizing Rebellion: Rethinking High-Risk Mobilization and Social Networks in War.” American Political Science Review 107 (03): 418–32.Google Scholar
Patty, John W., and Penn, Elizabeth Maggie. 2014. “Sequential Decision Making and Information Aggregation in Small Networks.” Political Science Research and Methods 2 (02): 243–71.Google Scholar
Putnam, Robert D. 2001. Bowling Alone: The Collapse and Revival of American Community. New York, NY: Simon & Schuster.Google Scholar
Rao, Neel, Mobius, Markus M., and Rosenblat, Tanya. 2007. “Social Networks and Vaccination Decisions.” Working Paper (Series Federal Reserve Bank of Boston No. 07-12). http://hdl.handle.net/10419/55601.Google Scholar
Salganik, Matthew J., Dodds, Peter Sheridan, and Watts, Duncan J.. 2006. “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.” Science 311 (5762): 854– 56.Google Scholar
Siegel, David A. 2009. “Social Networks and Collective Action.” American Journal of Political Science 53 (1): 122–38.Google Scholar
Sinclair, Betsy. 2012. The Social Citizen: Peer Networks and Political Behavior. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
Staniland, Paul. 2014. Networks of Rebellion: Explaining Insurgent Cohesion and Collapse. Ithaca, NY: Cornell University Press.Google Scholar
Steinert-Threlkeld, Zachary C. 2017. “Spontaneous Collective Action: Peripheral Mobilization during the Arab Spring.” American Political Science Review 111 (2): 379–403.Google Scholar
Steinert-Threlkeld, Zachary, Mocanu, Delia, Vespignani, Alessandro, and Fowler, James. 2015. “Online Social Networks and Offline Protest.” EPJ Data Science 4 (1): 1–9.Google Scholar
Stewart, Neil, Ungemach, Christoph, Harris, Adam J. L., Bartels, Daniel M., Newell, Ben R., Paolacci, Gabriele, and Chandler, Jesse. 2015. “The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers.” Judgment and Decision Making 10 (5): 479–91.Google Scholar
Subrahmanyam, Kaveri, Reich, Stephanie M., Waechter, Natalia, and Espinoza, Guadalupe. 2008. “Online and Offline Social Networks: Use of Social Networking Sites by Emerging Adults.” Journal of Applied Developmental Psychology 29 (6): 420–33.Google Scholar
Tezcür, Güneş Murat. 2016. “Ordinary People, Extraordinary Risks: Participation in an Ethnic Rebellion.” American Political Science Review 110 (02): 247–64.Google Scholar
Watt, Susan Ellen, Lea, Martin, Spears, Russell, and Rogers, Paul. 2002. How Social Is Internet Communication? Anonymity Effects in Computer-Mediated Groups. In Virtual Society? Technology, Cyberbole, Reality, ed. Woolgar, Steve. Oxford, UK: Oxford University Press, 61–77.Google Scholar
Weinberg, Jill D., Freese, Jeremy, and McElhattan, David. 2015. “Comparing Data Characteristics and Results of an Online Factorial Survey Between a Population-Based and a Crowdsource-Recruited Sample.” Sociological Science 1 (doi:10.15195/v1.a19): 292–310.Google Scholar
Westfall, Peter H., and Young, S. Stanley. 1993. Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment. Vol. 279. New York, NY: John Wiley & Sons.Google Scholar
Wilcox, Keith, and Stephen, Andrew T.. 2013. “Are Close Friends the Enemy? Online Social Networks, Self-Esteem, and Self-Control.” Journal of Consumer Research 40 (1): 90–103.Google Scholar
Wolitzky, Alexander. 2013. “Cooperation with Network Monitoring.” Review of Economic Studies 80 (1): 395–427.Google Scholar
Zimbardo, Philip G. 1969. “The Human Choice: Individuation, Reason, and Order Versus Deindividuation, Impulse, and Chaos.” In Nebraska Symposium on Motivation. Lincoln, NE: University of Nebraska Press, 237–307.Google Scholar
Supplementary material: PDF

Bisbee and Larson supplementary material

Online Appendix

Download Bisbee and Larson supplementary material(PDF)
PDF 302.3 KB